[RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameer Kolothum]

RFC v1 --> v2:
 - Added a generic interface for IOMMU drivers to retrieve all the 
   RMR info associated with a given IOMMU.
 - SMMUv3 driver gets the RMR list during probe() and installs
   bypass STEs for all the SIDs in the RMR list. This is to keep
   the ongoing traffic alive(if any) during SMMUv3 reset. This is
   based on the suggestions received for v1 to take care of the
   EFI framebuffer use case. Only sanity tested for now.
 - During the probe/attach device, SMMUv3 driver reserves any
   RMR region associated with the device such that there is a unity
   mapping for them in SMMU.
---    

The series adds support to IORT RMR nodes specified in IORT
Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
ranges that are used by endpoints and require a unity mapping
in SMMU.

We have faced issues with 3408iMR RAID controller cards which
fail to boot when SMMU is enabled. This is because these controllers
make use of host memory for various caching related purposes and when
SMMU is enabled the iMR firmware fails to access these memory regions
as there is no mapping for them. IORT RMR provides a way for UEFI to
describe and report these memory regions so that the kernel can make
a unity mapping for these in SMMU.

RFC because, Patch #1 is to update the actbl2.h and should be done
through acpica update. I have send out a pull request[1] for that.

Tests:

With a UEFI, that reports the RMR for the dev,
....
[16F0h 5872   1]                         Type : 06
[16F1h 5873   2]                       Length : 007C
[16F3h 5875   1]                     Revision : 00
[1038h 0056   2]                     Reserved : 00000000
[1038h 0056   2]                   Identifier : 00000000
[16F8h 5880   4]                Mapping Count : 00000001
[16FCh 5884   4]               Mapping Offset : 00000040

[1700h 5888   4]    Number of RMR Descriptors : 00000002
[1704h 5892   4]        RMR Descriptor Offset : 00000018

[1708h 5896   8]          Base Address of RMR : 0000E6400000
[1710h 5904   8]                Length of RMR : 000000100000
[1718h 5912   4]                     Reserved : 00000000

[171Ch 5916   8]          Base Address of RMR : 0000000027B00000
[1724h 5924   8]                Length of RMR : 0000000000C00000
[172Ch 5932   4]                     Reserved : 00000000

[1730h 5936   4]                   Input base : 00000000
[1734h 5940   4]                     ID Count : 00000001
[1738h 5944   4]                  Output Base : 00000003
[173Ch 5948   4]             Output Reference : 00000064
[1740h 5952   4]        Flags (decoded below) : 00000001
                               Single Mapping : 1
...

Without the series the RAID controller initialization fails as
below,

...
[   12.631117] megaraid_sas 0000:03:00.0: FW supports sync cache        : Yes   
[   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
[   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED for SCSI host 0                                                                         
[   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to ready state 
[  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault code:0x10000 subcode:0x0 func:megasas_transition_to_ready                                    
[  106.695186] megaraid_sas 0000:03:00.0: System Register set:                  
[  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller to ready for scsi0.                                                                   
[  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw 6407      
estuary:/$

With the series, now the kernel has direct mapping for the dev as
below,

estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions                      
0x0000000008000000 0x00000000080fffff msi                                       
0x0000000027b00000 0x00000000286fffff direct                                    
0x00000000e6400000 0x00000000e64fffff direct                                    
estuary:/$

....
[   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
[   12.739089] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs: 0      max_lds: 32                                                                      
[   12.746628] megaraid_sas 0000:03:00.0: controller type       : iMR(0MB)      
[   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  : Enabled                                                                               
[   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes           
[   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes           
[   12.771931] megaraid_sas 0000:03:00.0: FW provided TM TaskAbort/Reset timeou: 6 secs/60 secs                                                                 
[   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map support     : Yes   
[   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining support    : No    
[   12.819179] megaraid_sas 0000:03:00.0: NVME page size        : (4096)        
[   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is called outbound_intr_mask:0x40000000                                                    
[   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done                     
[   12.873932] megaraid_sas 0000:03:00.0: pci id                : (0x1000)/(0x0017)/(0x19e5)/(0xd213)                                                           
[   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no            
[   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no            
[   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     : enabled       

RAID controller init is now success and can detect the drives
attached as well.

Thanks,
Shameer

[0]. https://developer.arm.com/documentation/den0049/latest/
[1]. https://github.com/acpica/acpica/pull/638

Shameer Kolothum (8):
  ACPICA: IORT: Update for revision E
  ACPI/IORT: Add support for RMR node parsing
  iommu/dma: Introduce generic helper to retrieve RMR info
  ACPI/IORT: Add RMR memory regions reservation helper
  iommu/arm-smmu-v3: Introduce strtab init helper
  iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
  iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
  iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev

 drivers/acpi/arm64/iort.c                   | 182 +++++++++++++++++++-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 112 ++++++++++--
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   2 +
 drivers/iommu/dma-iommu.c                   |  39 +++++
 include/acpi/actbl2.h                       |  25 ++-
 include/linux/acpi_iort.h                   |   6 +
 include/linux/dma-iommu.h                   |   7 +
 include/linux/iommu.h                       |  16 ++
 8 files changed, 367 insertions(+), 22 deletions(-)

-- 
2.17.1

[RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Shameer Kolothum]

IORT revision E contains a few additions like,
    -Added an identifier field in the node descriptors to aid table
     cross-referencing.
    -Introduced the Reserved Memory Range(RMR) node. This is used
     to describe memory ranges that are used by endpoints and requires
     a unity mapping in SMMU.
    -Introduced a flag in the RC node to express support for PRI.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 include/acpi/actbl2.h | 25 +++++++++++++++++++------
 1 file changed, 19 insertions(+), 6 deletions(-)

diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h
index ec66779cb193..274fce7b5c01 100644
--- a/include/acpi/actbl2.h
+++ b/include/acpi/actbl2.h
@@ -68,7 +68,7 @@
  * IORT - IO Remapping Table
  *
  * Conforms to "IO Remapping Table System Software on ARM Platforms",
- * Document number: ARM DEN 0049D, March 2018
+ * Document number: ARM DEN 0049E, June 2020
  *
  ******************************************************************************/
 
@@ -86,7 +86,8 @@ struct acpi_iort_node {
 	u8 type;
 	u16 length;
 	u8 revision;
-	u32 reserved;
+	u16 reserved;
+	u16 identifier;
 	u32 mapping_count;
 	u32 mapping_offset;
 	char node_data[1];
@@ -100,7 +101,8 @@ enum acpi_iort_node_type {
 	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
 	ACPI_IORT_NODE_SMMU = 0x03,
 	ACPI_IORT_NODE_SMMU_V3 = 0x04,
-	ACPI_IORT_NODE_PMCG = 0x05
+	ACPI_IORT_NODE_PMCG = 0x05,
+	ACPI_IORT_NODE_RMR = 0x06,
 };
 
 struct acpi_iort_id_mapping {
@@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
 	u8 reserved[3];		/* Reserved, must be zero */
 };
 
-/* Values for ats_attribute field above */
+/* Masks for ats_attribute field above */
 
-#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root complex supports ATS */
-#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root complex doesn't support ATS */
+#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex supports ATS */
+#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root complex supports PRI */
 
 struct acpi_iort_smmu {
 	u64 base_address;	/* SMMU base address */
@@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
 	u64 page1_base_address;
 };
 
+struct acpi_iort_rmr {
+	u32 rmr_count;
+	u32 rmr_offset;
+};
+
+struct acpi_iort_rmr_desc {
+	u64 base_address;
+	u64 length;
+	u32 reserved;
+};
+
 /*******************************************************************************
  *
  * IVRS - I/O Virtualization Reporting Structure
-- 
2.17.1

[RFC PATCH v2 2/8] ACPI/IORT: Add support for RMR node parsing

[Shameer Kolothum]

Add support for parsing RMR node information from ACPI.
Find associated stream ids and smmu node info from the
RMR node and populate a linked list with RMR memory
descriptors.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 122 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 121 insertions(+), 1 deletion(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 9929ff50c0c0..a9705aa35028 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -40,6 +40,25 @@ struct iort_fwnode {
 static LIST_HEAD(iort_fwnode_list);
 static DEFINE_SPINLOCK(iort_fwnode_lock);
 
+struct iort_rmr_id {
+	u32  sid;
+	struct acpi_iort_node *smmu;
+};
+
+/*
+ * One entry for IORT RMR.
+ */
+struct iort_rmr_entry {
+	struct list_head list;
+
+	unsigned int rmr_ids_num;
+	struct iort_rmr_id *rmr_ids;
+
+	struct acpi_iort_rmr_desc *rmr_desc;
+};
+
+static LIST_HEAD(iort_rmr_list);         /* list of RMR regions from ACPI */
+
 /**
  * iort_set_fwnode() - Create iort_fwnode and use it to register
  *		       iommu data in the iort_fwnode_list
@@ -393,7 +412,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
 		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
 		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
 		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
-		    node->type == ACPI_IORT_NODE_PMCG) {
+		    node->type == ACPI_IORT_NODE_PMCG ||
+		    node->type == ACPI_IORT_NODE_RMR) {
 			*id_out = map->output_base;
 			return parent;
 		}
@@ -1647,6 +1667,103 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
 #else
 static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
 #endif
+static int iort_rmr_desc_valid(struct acpi_iort_rmr_desc *desc)
+{
+	struct iort_rmr_entry *e;
+	u64 end, start = desc->base_address, length = desc->length;
+
+	if (!IS_ALIGNED(start, SZ_64K) || !IS_ALIGNED(length, SZ_64K))
+		return -EINVAL;
+
+	end = start + length - 1;
+
+	/* Check for address overlap */
+	list_for_each_entry(e, &iort_rmr_list, list) {
+		u64 e_start = e->rmr_desc->base_address;
+		u64 e_end = e_start + e->rmr_desc->length - 1;
+
+		if (start <= e_end && end >= e_start)
+			return -EINVAL;
+	}
+
+	return 0;
+}
+
+static int __init iort_parse_rmr(struct acpi_iort_node *iort_node)
+{
+	struct iort_rmr_id *rmr_ids, *ids;
+	struct iort_rmr_entry *e;
+	struct acpi_iort_rmr *rmr;
+	struct acpi_iort_rmr_desc *rmr_desc;
+	u32 map_count = iort_node->mapping_count;
+	int i, ret = 0, desc_count = 0;
+
+	if (iort_node->type != ACPI_IORT_NODE_RMR)
+		return 0;
+
+	if (!iort_node->mapping_offset || !map_count) {
+		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
+		       iort_node);
+		return -EINVAL;
+	}
+
+	rmr_ids = kmalloc(sizeof(*rmr_ids) * map_count, GFP_KERNEL);
+	if (!rmr_ids)
+		return -ENOMEM;
+
+	/* Retrieve associated smmu and stream id */
+	ids = rmr_ids;
+	for (i = 0; i < map_count; i++, ids++) {
+		ids->smmu = iort_node_get_id(iort_node, &ids->sid, i);
+		if (!ids->smmu) {
+			pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
+			       iort_node);
+			ret = -EINVAL;
+			goto out;
+		}
+	}
+
+	/* Retrieve RMR data */
+	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
+	if (!rmr->rmr_offset || !rmr->rmr_count) {
+		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
+		       iort_node);
+		ret = -EINVAL;
+		goto out;
+	}
+
+	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
+				rmr->rmr_offset);
+
+	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
+		ret = iort_rmr_desc_valid(rmr_desc);
+		if (ret) {
+			pr_err(FW_BUG "Invalid RMR descriptor[%d] for node %p, skipping...\n",
+			       i, iort_node);
+			goto out;
+		}
+
+		e = kmalloc(sizeof(*e), GFP_KERNEL);
+		if (!e) {
+			ret = -ENOMEM;
+			goto out;
+		}
+
+		e->rmr_ids_num = map_count;
+		e->rmr_ids = rmr_ids;
+		e->rmr_desc = rmr_desc;
+
+		list_add_tail(&e->list, &iort_rmr_list);
+		desc_count++;
+	}
+
+	return 0;
+
+out:
+	if (!desc_count)
+		kfree(rmr_ids);
+	return ret;
+}
 
 static void __init iort_init_platform_devices(void)
 {
@@ -1676,6 +1793,9 @@ static void __init iort_init_platform_devices(void)
 
 		iort_enable_acs(iort_node);
 
+		if (iort_table->revision == 1)
+			iort_parse_rmr(iort_node);
+
 		ops = iort_get_dev_cfg(iort_node);
 		if (ops) {
 			fwnode = acpi_alloc_fwnode_static();
-- 
2.17.1

[RFC PATCH v2 3/8] iommu/dma: Introduce generic helper to retrieve RMR info

[Shameer Kolothum]

Reserved Memory Regions(RMR) associated with an IOMMU may be
described either through ACPI tables or DT in systems with
devices that require a unity mapping or bypass for those
regions in IOMMU drivers.

Introduce a generic interface so that IOMMU drivers can retrieve
and set up necessary mappings.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/dma-iommu.c | 36 ++++++++++++++++++++++++++++++++++++
 include/linux/dma-iommu.h |  7 +++++++
 include/linux/iommu.h     | 16 ++++++++++++++++
 3 files changed, 59 insertions(+)

diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 0cbcd3fc3e7e..d73768ecdd1a 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -166,6 +166,42 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
+/**
+ * iommu_dma_get_rmrs - Retrieve Reserved Memory Regions(RMRs) associated
+ *                      with a given IOMMU
+ * @iommu_fwnode: fwnode associated with IOMMU
+ * @list: RMR list to be populated
+ *
+ */
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
+		       struct list_head *list)
+{
+	return 0;
+}
+EXPORT_SYMBOL(iommu_dma_get_rmrs);
+
+struct iommu_rmr *iommu_dma_alloc_rmr(u64 base, u64 length,
+				      u32 *ids, int num_ids)
+{
+	struct iommu_rmr *rmr;
+	int i;
+
+	rmr = kzalloc(struct_size(rmr, ids, num_ids), GFP_KERNEL);
+	if (!rmr)
+		return NULL;
+
+	INIT_LIST_HEAD(&rmr->list);
+	rmr->base_address = base;
+	rmr->length = length;
+	rmr->num_ids = num_ids;
+
+	for (i = 0; i < num_ids; i++)
+		rmr->ids[i] = ids[i];
+
+	return rmr;
+}
+EXPORT_SYMBOL(iommu_dma_alloc_rmr);
+
 static int cookie_init_hw_msi_region(struct iommu_dma_cookie *cookie,
 		phys_addr_t start, phys_addr_t end)
 {
diff --git a/include/linux/dma-iommu.h b/include/linux/dma-iommu.h
index 2112f21f73d8..8900ccbc9e6a 100644
--- a/include/linux/dma-iommu.h
+++ b/include/linux/dma-iommu.h
@@ -37,6 +37,9 @@ void iommu_dma_compose_msi_msg(struct msi_desc *desc,
 
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
 
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+struct iommu_rmr *iommu_dma_alloc_rmr(u64 base, u64 length,
+				      u32 *ids, int num_ids);
 #else /* CONFIG_IOMMU_DMA */
 
 struct iommu_domain;
@@ -78,5 +81,9 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+int iommu_dma_get_rmrs(struct fwnode_handle *iommu, struct list_head *list);
+{
+	return 0;
+}
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index b95a6f8db6ff..e43c4e8084e7 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -592,6 +592,22 @@ struct iommu_sva {
 	struct device			*dev;
 };
 
+/**
+ * struct iommu_rmr - Reserved Memory Region details per IOMMU
+ * @list: Linked list pointers to hold RMR region info
+ * @base_address: base address of Reserved Memory Region
+ * @length: length of memory region
+ * @num_ids: number of associated device IDs
+ * @ids: associated device IDs
+ */
+struct iommu_rmr {
+	struct list_head	list;
+	phys_addr_t		base_address;
+	u64			length;
+	unsigned int		num_ids;
+	u32			ids[];
+};
+
 int iommu_fwspec_init(struct device *dev, struct fwnode_handle *iommu_fwnode,
 		      const struct iommu_ops *ops);
 void iommu_fwspec_free(struct device *dev);
-- 
2.17.1

[RFC PATCH v2 4/8] ACPI/IORT: Add RMR memory regions reservation helper

[Shameer Kolothum]

Add a helper function that retrieves RMR memory descriptors
associated with a given IOMMU. This will be used by IOMMU
drivers to setup necessary mappings.

Now that we have this, invoke this from the generic helper
interface.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c | 60 +++++++++++++++++++++++++++++++++++++++
 drivers/iommu/dma-iommu.c |  3 ++
 include/linux/acpi_iort.h |  6 ++++
 3 files changed, 69 insertions(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index a9705aa35028..d1a2b09230ab 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -12,6 +12,7 @@
 
 #include <linux/acpi_iort.h>
 #include <linux/bitfield.h>
+#include <linux/dma-iommu.h>
 #include <linux/iommu.h>
 #include <linux/kernel.h>
 #include <linux/list.h>
@@ -842,6 +843,63 @@ static inline int iort_add_device_replay(struct device *dev)
 	return err;
 }
 
+/**
+ * iort_iommu_get_rmrs - Helper to retrieve RMR info associated with IOMMU
+ * @iommu: fwnode for the IOMMU
+ * @head: RMR list head to be populated
+ *
+ * Returns: 0 on success, <0 failure
+ */
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *head)
+{
+	struct iort_rmr_entry *e;
+	struct acpi_iort_node *iommu;
+
+	iommu = iort_get_iort_node(iommu_fwnode);
+	if (!iommu)
+		return 0;
+
+	list_for_each_entry(e, &iort_rmr_list, list) {
+		struct iort_rmr_id *rmr_ids = e->rmr_ids;
+		struct acpi_iort_rmr_desc *rmr_desc;
+		struct iommu_rmr *rmr;
+		u32 *ids, num_ids = 0;
+		int i, j = 0;
+
+		for (i = 0; i < e->rmr_ids_num; i++) {
+			if (rmr_ids[i].smmu == iommu)
+				num_ids++;
+		}
+
+		if (!num_ids)
+			continue;
+
+		ids = kmalloc_array(num_ids, sizeof(*ids), GFP_KERNEL);
+		if (!ids)
+			return -ENOMEM;
+
+		for (i = 0; i < e->rmr_ids_num; i++) {
+			if (rmr_ids[i].smmu == iommu)
+				ids[j++] = rmr_ids[i].sid;
+		}
+
+		rmr_desc = e->rmr_desc;
+		rmr = iommu_dma_alloc_rmr(rmr_desc->base_address,
+					  rmr_desc->length,
+					  ids, num_ids);
+		if (!rmr) {
+			kfree(ids);
+			return -ENOMEM;
+		}
+
+		list_add_tail(&rmr->list, head);
+		kfree(ids);
+	}
+
+	return 0;
+}
+
 /**
  * iort_iommu_msi_get_resv_regions - Reserved region driver helper
  * @dev: Device from iommu_get_resv_regions()
@@ -1112,6 +1170,8 @@ int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
 						const u32 *input_id)
 { return NULL; }
+int iort_iommu_get_rmrs(struct fwnode_handle *fwnode, struct list_head *head)
+{ return 0; }
 #endif
 
 static int nc_dma_get_range(struct device *dev, u64 *size)
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index d73768ecdd1a..aa8304e50786 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -176,6 +176,9 @@ EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 int iommu_dma_get_rmrs(struct fwnode_handle *iommu_fwnode,
 		       struct list_head *list)
 {
+	if (!is_of_node(iommu_fwnode))
+		return iort_iommu_get_rmrs(iommu_fwnode, list);
+
 	return 0;
 }
 EXPORT_SYMBOL(iommu_dma_get_rmrs);
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 20a32120bb88..0b61b98a4941 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -38,6 +38,8 @@ void iort_dma_setup(struct device *dev, u64 *dma_addr, u64 *size);
 const struct iommu_ops *iort_iommu_configure_id(struct device *dev,
 						const u32 *id_in);
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head);
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list);
 #else
 static inline void acpi_iort_init(void) { }
 static inline u32 iort_msi_map_id(struct device *dev, u32 id)
@@ -55,6 +57,10 @@ static inline const struct iommu_ops *iort_iommu_configure_id(
 static inline
 int iort_iommu_msi_get_resv_regions(struct device *dev, struct list_head *head)
 { return 0; }
+static inline
+int iort_iommu_get_rmrs(struct fwnode_handle *iommu_fwnode,
+			struct list_head *list)
+{ return 0; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
2.17.1

[RFC PATCH v2 5/8] iommu/arm-smmu-v3: Introduce strtab init helper

[Shameer Kolothum]

Introduce a helper to check the sid range and to init the l2 strtab
entries(bypass). This will be useful when we have to initialize the
l2 strtab for RMR SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 26 ++++++++++++---------
 1 file changed, 15 insertions(+), 11 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index e634bbe60573..1953b317d814 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2308,6 +2308,19 @@ static bool arm_smmu_sid_in_range(struct arm_smmu_device *smmu, u32 sid)
 
 static struct iommu_ops arm_smmu_ops;
 
+static int arm_smmu_init_sid_strtab(struct arm_smmu_device *smmu, u32 sid)
+{
+	/* Check the SIDs are in range of the SMMU and our stream table */
+	if (!arm_smmu_sid_in_range(smmu, sid))
+		return -ERANGE;
+
+	/* Ensure l2 strtab is initialised */
+	if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB)
+		return arm_smmu_init_l2_strtab(smmu, sid);
+
+	return 0;
+}
+
 static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 {
 	int i, ret;
@@ -2336,21 +2349,12 @@ static struct iommu_device *arm_smmu_probe_device(struct device *dev)
 	INIT_LIST_HEAD(&master->bonds);
 	dev_iommu_priv_set(dev, master);
 
-	/* Check the SIDs are in range of the SMMU and our stream table */
 	for (i = 0; i < master->num_sids; i++) {
 		u32 sid = master->sids[i];
 
-		if (!arm_smmu_sid_in_range(smmu, sid)) {
-			ret = -ERANGE;
+		ret = arm_smmu_init_sid_strtab(smmu, sid);
+		if (ret)
 			goto err_free_master;
-		}
-
-		/* Ensure l2 strtab is initialised */
-		if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) {
-			ret = arm_smmu_init_l2_strtab(smmu, sid);
-			if (ret)
-				goto err_free_master;
-		}
 	}
 
 	master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits);
-- 
2.17.1

[RFC PATCH v2 6/8] iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()

[Shameer Kolothum]

By default, disable_bypass is set and any dev without an iommu domain
installs STE with CFG_ABORT during arm_smmu_init_bypass_stes(). Introduce
a "bypass" flag to arm_smmu_write_strtab_ent() so that we can force it to
install CFG_BYPASS STE for specific SIDs. This will be useful for RMR
related SIDs.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 1953b317d814..5f366d5a9ebf 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1174,7 +1174,7 @@ static void arm_smmu_sync_ste_for_sid(struct arm_smmu_device *smmu, u32 sid)
 }
 
 static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
-				      __le64 *dst)
+				      __le64 *dst, bool bypass)
 {
 	/*
 	 * This is hideously complicated, but we only really care about
@@ -1245,7 +1245,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
 
 	/* Bypass/fault */
 	if (!smmu_domain || !(s1_cfg || s2_cfg)) {
-		if (!smmu_domain && disable_bypass)
+		if (!smmu_domain && disable_bypass && !bypass)
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_ABORT);
 		else
 			val |= FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_BYPASS);
@@ -1317,7 +1317,7 @@ static void arm_smmu_init_bypass_stes(__le64 *strtab, unsigned int nent)
 	unsigned int i;
 
 	for (i = 0; i < nent; ++i) {
-		arm_smmu_write_strtab_ent(NULL, -1, strtab);
+		arm_smmu_write_strtab_ent(NULL, -1, strtab, false);
 		strtab += STRTAB_STE_DWORDS;
 	}
 }
@@ -2038,7 +2038,7 @@ static void arm_smmu_install_ste_for_dev(struct arm_smmu_master *master)
 		if (j < i)
 			continue;
 
-		arm_smmu_write_strtab_ent(master, sid, step);
+		arm_smmu_write_strtab_ent(master, sid, step, false);
 	}
 }
 
-- 
2.17.1

[RFC PATCH v2 7/8] iommu/arm-smmu-v3: Get associated RMR info and install bypass STE

[Shameer Kolothum]

Check if there is any RMR info associated with the devices behind
the SMMUv3 and if any, install bypass STEs for them. This is to
keep any ongoing traffic associated with these devices alive
when we enable/reset SMMUv3 during probe().

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 40 +++++++++++++++++++++
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |  2 ++
 2 files changed, 42 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 5f366d5a9ebf..97df1df001c9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -3486,6 +3486,42 @@ static void __iomem *arm_smmu_ioremap(struct device *dev, resource_size_t start,
 	return devm_ioremap_resource(dev, &res);
 }
 
+static void arm_smmu_rmr_install_bypass_ste(struct arm_smmu_device *smmu)
+{
+	struct iommu_rmr *e;
+	int i, ret;
+
+	/*
+	 * Since, we don't have a mechanism to differentiate the RMR
+	 * SIDs that has an ongoing live stream, install bypass STEs
+	 * for all the reported ones. 
+	 * FixMe: Avoid duplicate SIDs in the list as one sid may
+	 *        associate with multiple RMRs.
+	 */
+	list_for_each_entry(e, &smmu->rmr_list, list) {
+		for (i = 0; i < e->num_ids; i++) {
+			__le64 *step;
+			u32 sid = e->ids[i];
+
+			ret = arm_smmu_init_sid_strtab(smmu, sid);
+			if (ret) {
+				dev_err(smmu->dev, "RMR bypass(0x%x) failed\n",
+					sid);
+				continue;
+			}
+
+			step = arm_smmu_get_step_for_sid(smmu, sid);
+			arm_smmu_write_strtab_ent(NULL, sid, step, true);
+		}
+	}
+}
+
+static int arm_smmu_get_rmr(struct arm_smmu_device *smmu)
+{
+	INIT_LIST_HEAD(&smmu->rmr_list);
+	return iommu_dma_get_rmrs(dev_fwnode(smmu->dev), &smmu->rmr_list);
+}
+
 static int arm_smmu_device_probe(struct platform_device *pdev)
 {
 	int irq, ret;
@@ -3569,6 +3605,10 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	/* Record our private device structure */
 	platform_set_drvdata(pdev, smmu);
 
+	/* Check for RMRs and install bypass STEs if any */
+	if (!arm_smmu_get_rmr(smmu))
+		arm_smmu_rmr_install_bypass_ste(smmu);
+
 	/* Reset the device */
 	ret = arm_smmu_device_reset(smmu, bypass);
 	if (ret)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index d4b7f40ccb02..17b517ddecee 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -636,6 +636,8 @@ struct arm_smmu_device {
 
 	/* IOMMU core code handle */
 	struct iommu_device		iommu;
+
+	struct list_head		rmr_list;
 };
 
 /* SMMU private data for each master */
-- 
2.17.1

[RFC PATCH v2 8/8] iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev

[Shameer Kolothum]

Get RMR regions associated with a dev reserved so that there is
a unity mapping for them in SMMU.

Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 38 +++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 97df1df001c9..174a9bcfd627 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2492,6 +2492,43 @@ static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
 	return iommu_fwspec_add_ids(dev, args->args, 1);
 }
 
+static bool arm_smmu_dev_has_rmr(struct arm_smmu_master *master,
+				 struct iommu_rmr *e)
+{
+	int i, j;
+
+	for (i = 0; i < master->num_sids; i++) {
+		for (j = 0; j < e->num_ids; j++) {
+			if (e->ids[j] == master->sids[i])
+				return true;
+		}
+	}
+
+	return false;
+}
+
+static void arm_smmu_rmr_get_resv_regions(struct device *dev,
+					  struct list_head *head)
+{
+	struct arm_smmu_master *master = dev_iommu_priv_get(dev);
+	struct arm_smmu_device *smmu = master->smmu;
+	struct iommu_rmr *rmr;
+
+	list_for_each_entry(rmr, &smmu->rmr_list, list) {
+		struct iommu_resv_region *region;
+		int prot = IOMMU_READ | IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+
+		if (!arm_smmu_dev_has_rmr(master, rmr))
+			continue;
+		region = iommu_alloc_resv_region(rmr->base_address,
+						 rmr->length, prot,
+						 IOMMU_RESV_DIRECT);
+		if (!region)
+			return;
+
+		list_add_tail(&region->list, head);
+	}
+}
 static void arm_smmu_get_resv_regions(struct device *dev,
 				      struct list_head *head)
 {
@@ -2506,6 +2543,7 @@ static void arm_smmu_get_resv_regions(struct device *dev,
 	list_add_tail(&region->list, head);
 
 	iommu_dma_get_resv_regions(dev, head);
+	arm_smmu_rmr_get_resv_regions(dev, head);
 }
 
 static bool arm_smmu_dev_has_feature(struct device *dev,
-- 
2.17.1

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Steven Price]

On 19/11/2020 12:11, Shameer Kolothum wrote:
> RFC v1 --> v2:
>   - Added a generic interface for IOMMU drivers to retrieve all the
>     RMR info associated with a given IOMMU.
>   - SMMUv3 driver gets the RMR list during probe() and installs
>     bypass STEs for all the SIDs in the RMR list. This is to keep
>     the ongoing traffic alive(if any) during SMMUv3 reset. This is
>     based on the suggestions received for v1 to take care of the
>     EFI framebuffer use case. Only sanity tested for now.

Hi Shameer,

Sorry for not looking at this before.

Do you have any plans to implement support in the SMMUv2 driver? The 
platform I've been testing the EFI framebuffer support on has the 
display controller behind SMMUv2, so as it stands this series doesn't 
work. I did hack something up for SMMUv2 so I was able to test the first 
4 patches.

>   - During the probe/attach device, SMMUv3 driver reserves any
>     RMR region associated with the device such that there is a unity
>     mapping for them in SMMU.

For the EFI framebuffer use case there is no device to attach so I 
believe we are left with just the stream ID in bypass mode - which is 
definitely an improvement (the display works!) but not actually a unity 
mapping of the RMR range. I'm not sure whether it's worth fixing this or 
not, but I just wanted to point out there's still a need for a driver 
for the device before the bypass mode is replaced with the unity mapping.

Thanks,

Steve

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

Hi Steve,

> -----Original Message-----
> From: Steven Price [mailto:steven.price@arm.com]
> Sent: 10 December 2020 10:26
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> <guohanjun@huawei.com>; Jonathan Cameron
> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > RFC v1 --> v2:
> >   - Added a generic interface for IOMMU drivers to retrieve all the
> >     RMR info associated with a given IOMMU.
> >   - SMMUv3 driver gets the RMR list during probe() and installs
> >     bypass STEs for all the SIDs in the RMR list. This is to keep
> >     the ongoing traffic alive(if any) during SMMUv3 reset. This is
> >     based on the suggestions received for v1 to take care of the
> >     EFI framebuffer use case. Only sanity tested for now.
> 
> Hi Shameer,
> 
> Sorry for not looking at this before.
> 
> Do you have any plans to implement support in the SMMUv2 driver? The
> platform I've been testing the EFI framebuffer support on has the
> display controller behind SMMUv2, so as it stands this series doesn't
> work. I did hack something up for SMMUv2 so I was able to test the first
> 4 patches.

Thanks for taking a look. Sure, I can look into adding the support for SMMUv2. 

> 
> >   - During the probe/attach device, SMMUv3 driver reserves any
> >     RMR region associated with the device such that there is a unity
> >     mapping for them in SMMU.
> 
> For the EFI framebuffer use case there is no device to attach so I
> believe we are left with just the stream ID in bypass mode - which is
> definitely an improvement (the display works!)

Cool. That’s good to know.

 but not actually a unity
> mapping of the RMR range. I'm not sure whether it's worth fixing this or
> not, but I just wanted to point out there's still a need for a driver
> for the device before the bypass mode is replaced with the unity mapping.

I am not sure either. My idea was we will have bypass STE setup for all devices
with RMR initially and when the corresponding driver takes over(if that happens)
we will have the unity mapping setup properly for the RMR regions. And for cases
like the above, it will remain in the bypass mode.

Do you see any problem(security?) if the dev streams remain in bypass mode for
this dev? Or is it possible to have a stub driver for this dev, so that we will have
the probe/attach invoked and everything will fall in place?

TBH, I haven't looked into creating a temp domain for these types of the devices
and also not sure how we benefit from that compared to the STE bypass mode.

Thoughts/Ideas welcome.

Thanks,
Shameer

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Robin Murphy]

On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> Hi Steve,
> 
>> -----Original Message-----
>> From: Steven Price [mailto:steven.price@arm.com]
>> Sent: 10 December 2020 10:26
>> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org; devel@acpica.org
>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>> <guohanjun@huawei.com>; Jonathan Cameron
>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
>>
>> On 19/11/2020 12:11, Shameer Kolothum wrote:
>>> RFC v1 --> v2:
>>>    - Added a generic interface for IOMMU drivers to retrieve all the
>>>      RMR info associated with a given IOMMU.
>>>    - SMMUv3 driver gets the RMR list during probe() and installs
>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
>>>      based on the suggestions received for v1 to take care of the
>>>      EFI framebuffer use case. Only sanity tested for now.
>>
>> Hi Shameer,
>>
>> Sorry for not looking at this before.
>>
>> Do you have any plans to implement support in the SMMUv2 driver? The
>> platform I've been testing the EFI framebuffer support on has the
>> display controller behind SMMUv2, so as it stands this series doesn't
>> work. I did hack something up for SMMUv2 so I was able to test the first
>> 4 patches.
> 
> Thanks for taking a look. Sure, I can look into adding the support for SMMUv2.
> 
>>
>>>    - During the probe/attach device, SMMUv3 driver reserves any
>>>      RMR region associated with the device such that there is a unity
>>>      mapping for them in SMMU.
>>
>> For the EFI framebuffer use case there is no device to attach so I
>> believe we are left with just the stream ID in bypass mode - which is
>> definitely an improvement (the display works!)
> 
> Cool. That’s good to know.
> 
>   but not actually a unity
>> mapping of the RMR range. I'm not sure whether it's worth fixing this or
>> not, but I just wanted to point out there's still a need for a driver
>> for the device before the bypass mode is replaced with the unity mapping.
> 
> I am not sure either. My idea was we will have bypass STE setup for all devices
> with RMR initially and when the corresponding driver takes over(if that happens)
> we will have the unity mapping setup properly for the RMR regions. And for cases
> like the above, it will remain in the bypass mode.
> 
> Do you see any problem(security?) if the dev streams remain in bypass mode for
> this dev? Or is it possible to have a stub driver for this dev, so that we will have
> the probe/attach invoked and everything will fall in place?

The downside is that if a driver never binds to that device, it remains 
bypassed. If some other externally-controlled malicious device could 
manage to spoof that device's requester ID, that could potentially be 
exploited.

> TBH, I haven't looked into creating a temp domain for these types of the devices
> and also not sure how we benefit from that compared to the STE bypass mode.

That said, setting up temporary translation contexts that ensure any 
access can *only* be to RMR regions until a driver takes over is an 
awful lot more work. I'm inclined to be pragmatic here and say we should 
get things working at all with the simple bypass STE/S2CR method, then 
look at adding the higher-security effort on top.

Right now systems that need this are either broken (but effectively 
secure) or using default bypass with SMMUv2. People who prefer to 
maintain security over functionality in the interim can maintain that 
status quo by simply continuing to not describe any RMRs.

Robin.

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Steven Price]

On 14/12/2020 12:33, Robin Murphy wrote:
> On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
>> Hi Steve,
>>
>>> -----Original Message-----
>>> From: Steven Price [mailto:steven.price@arm.com]
>>> Sent: 10 December 2020 10:26
>>> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
>>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
>>> iommu@lists.linux-foundation.org; devel@acpica.org
>>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
>>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
>>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
>>> <guohanjun@huawei.com>; Jonathan Cameron
>>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
>>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
>>>
>>> On 19/11/2020 12:11, Shameer Kolothum wrote:
>>>> RFC v1 --> v2:
>>>>    - Added a generic interface for IOMMU drivers to retrieve all the
>>>>      RMR info associated with a given IOMMU.
>>>>    - SMMUv3 driver gets the RMR list during probe() and installs
>>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
>>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
>>>>      based on the suggestions received for v1 to take care of the
>>>>      EFI framebuffer use case. Only sanity tested for now.
>>>
>>> Hi Shameer,
>>>
>>> Sorry for not looking at this before.
>>>
>>> Do you have any plans to implement support in the SMMUv2 driver? The
>>> platform I've been testing the EFI framebuffer support on has the
>>> display controller behind SMMUv2, so as it stands this series doesn't
>>> work. I did hack something up for SMMUv2 so I was able to test the first
>>> 4 patches.
>>
>> Thanks for taking a look. Sure, I can look into adding the support for 
>> SMMUv2.

Great, thanks!

>>>
>>>>    - During the probe/attach device, SMMUv3 driver reserves any
>>>>      RMR region associated with the device such that there is a unity
>>>>      mapping for them in SMMU.
>>>
>>> For the EFI framebuffer use case there is no device to attach so I
>>> believe we are left with just the stream ID in bypass mode - which is
>>> definitely an improvement (the display works!)
>>
>> Cool. That’s good to know.
>>
>>   but not actually a unity
>>> mapping of the RMR range. I'm not sure whether it's worth fixing this or
>>> not, but I just wanted to point out there's still a need for a driver
>>> for the device before the bypass mode is replaced with the unity 
>>> mapping.
>>
>> I am not sure either. My idea was we will have bypass STE setup for 
>> all devices
>> with RMR initially and when the corresponding driver takes over(if 
>> that happens)
>> we will have the unity mapping setup properly for the RMR regions. And 
>> for cases
>> like the above, it will remain in the bypass mode.
>>
>> Do you see any problem(security?) if the dev streams remain in bypass 
>> mode for
>> this dev? Or is it possible to have a stub driver for this dev, so 
>> that we will have
>> the probe/attach invoked and everything will fall in place?
> 
> The downside is that if a driver never binds to that device, it remains 
> bypassed. If some other externally-controlled malicious device could 
> manage to spoof that device's requester ID, that could potentially be 
> exploited.
> 
>> TBH, I haven't looked into creating a temp domain for these types of 
>> the devices
>> and also not sure how we benefit from that compared to the STE bypass 
>> mode.
> 
> That said, setting up temporary translation contexts that ensure any 
> access can *only* be to RMR regions until a driver takes over is an 
> awful lot more work. I'm inclined to be pragmatic here and say we should 
> get things working at all with the simple bypass STE/S2CR method, then 
> look at adding the higher-security effort on top.
> 
> Right now systems that need this are either broken (but effectively 
> secure) or using default bypass with SMMUv2. People who prefer to 
> maintain security over functionality in the interim can maintain that 
> status quo by simply continuing to not describe any RMRs.

I agree with Robin, let's get this working with the bypass mode (until 
the device binds) like you've currently got. It's much better than what 
we have otherwise. Then once that is merged we can look at the temporary 
translation context or stub driver. The temporary translation context 
would be 'neatest', but I'm only aware of the EFI framebuffer use case 
and for that it might be possible to do something simpler - if indeed 
anything is needed at all. I'm not sure how much we need to be worried 
about malicious devices spoofing requester IDs.

Thanks,

Steve

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Steven Price [mailto:steven.price@arm.com]
> Sent: 14 December 2020 13:43
> To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> On 14/12/2020 12:33, Robin Murphy wrote:
> > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> >> Hi Steve,
> >>
> >>> -----Original Message-----
> >>> From: Steven Price [mailto:steven.price@arm.com]
> >>> Sent: 10 December 2020 10:26
> >>> To: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>;
> >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> >>> iommu@lists.linux-foundation.org; devel@acpica.org
> >>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> >>> <guohanjun@huawei.com>; Jonathan Cameron
> >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> >>>
> >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> >>>> RFC v1 --> v2:
> >>>>    - Added a generic interface for IOMMU drivers to retrieve all the
> >>>>      RMR info associated with a given IOMMU.
> >>>>    - SMMUv3 driver gets the RMR list during probe() and installs
> >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
> >>>>      based on the suggestions received for v1 to take care of the
> >>>>      EFI framebuffer use case. Only sanity tested for now.
> >>>
> >>> Hi Shameer,
> >>>
> >>> Sorry for not looking at this before.
> >>>
> >>> Do you have any plans to implement support in the SMMUv2 driver? The
> >>> platform I've been testing the EFI framebuffer support on has the
> >>> display controller behind SMMUv2, so as it stands this series doesn't
> >>> work. I did hack something up for SMMUv2 so I was able to test the first
> >>> 4 patches.
> >>
> >> Thanks for taking a look. Sure, I can look into adding the support for
> >> SMMUv2.
> 
> Great, thanks!
> 
> >>>
> >>>>    - During the probe/attach device, SMMUv3 driver reserves any
> >>>>      RMR region associated with the device such that there is a unity
> >>>>      mapping for them in SMMU.
> >>>
> >>> For the EFI framebuffer use case there is no device to attach so I
> >>> believe we are left with just the stream ID in bypass mode - which is
> >>> definitely an improvement (the display works!)
> >>
> >> Cool. That’s good to know.
> >>
> >>   but not actually a unity
> >>> mapping of the RMR range. I'm not sure whether it's worth fixing this or
> >>> not, but I just wanted to point out there's still a need for a driver
> >>> for the device before the bypass mode is replaced with the unity
> >>> mapping.
> >>
> >> I am not sure either. My idea was we will have bypass STE setup for
> >> all devices
> >> with RMR initially and when the corresponding driver takes over(if
> >> that happens)
> >> we will have the unity mapping setup properly for the RMR regions. And
> >> for cases
> >> like the above, it will remain in the bypass mode.
> >>
> >> Do you see any problem(security?) if the dev streams remain in bypass
> >> mode for
> >> this dev? Or is it possible to have a stub driver for this dev, so
> >> that we will have
> >> the probe/attach invoked and everything will fall in place?
> >
> > The downside is that if a driver never binds to that device, it remains
> > bypassed. If some other externally-controlled malicious device could
> > manage to spoof that device's requester ID, that could potentially be
> > exploited.
> >
> >> TBH, I haven't looked into creating a temp domain for these types of
> >> the devices
> >> and also not sure how we benefit from that compared to the STE bypass
> >> mode.
> >
> > That said, setting up temporary translation contexts that ensure any
> > access can *only* be to RMR regions until a driver takes over is an
> > awful lot more work. I'm inclined to be pragmatic here and say we should
> > get things working at all with the simple bypass STE/S2CR method, then
> > look at adding the higher-security effort on top.
> >
> > Right now systems that need this are either broken (but effectively
> > secure) or using default bypass with SMMUv2. People who prefer to
> > maintain security over functionality in the interim can maintain that
> > status quo by simply continuing to not describe any RMRs.
> 
> I agree with Robin, let's get this working with the bypass mode (until
> the device binds) like you've currently got. It's much better than what
> we have otherwise. Then once that is merged we can look at the temporary
> translation context or stub driver. The temporary translation context
> would be 'neatest', but I'm only aware of the EFI framebuffer use case
> and for that it might be possible to do something simpler - if indeed
> anything is needed at all. I'm not sure how much we need to be worried
> about malicious devices spoofing requester IDs.

Perfect. I will keep the STE bypass and respin the series once the update
to the IORT rev E is made public(hope that will happen soon). In the
meantime, appreciate any feedback on the rest of the patches in this series.

Thanks,
Shameer

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Jon Nettleton]

On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Steven Price [mailto:steven.price@arm.com]
> > Sent: 14 December 2020 13:43
> > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>;
> > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-foundation.org; devel@acpica.org
> > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> >
> > On 14/12/2020 12:33, Robin Murphy wrote:
> > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > >> Hi Steve,
> > >>
> > >>> -----Original Message-----
> > >>> From: Steven Price [mailto:steven.price@arm.com]
> > >>> Sent: 10 December 2020 10:26
> > >>> To: Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>;
> > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > >>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > >>>
> > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > >>>> RFC v1 --> v2:
> > >>>>    - Added a generic interface for IOMMU drivers to retrieve all the
> > >>>>      RMR info associated with a given IOMMU.
> > >>>>    - SMMUv3 driver gets the RMR list during probe() and installs
> > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
> > >>>>      based on the suggestions received for v1 to take care of the
> > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > >>>
> > >>> Hi Shameer,
> > >>>
> > >>> Sorry for not looking at this before.
> > >>>
> > >>> Do you have any plans to implement support in the SMMUv2 driver? The
> > >>> platform I've been testing the EFI framebuffer support on has the
> > >>> display controller behind SMMUv2, so as it stands this series doesn't
> > >>> work. I did hack something up for SMMUv2 so I was able to test the first
> > >>> 4 patches.
> > >>
> > >> Thanks for taking a look. Sure, I can look into adding the support for
> > >> SMMUv2.
> >
> > Great, thanks!
> >
> > >>>
> > >>>>    - During the probe/attach device, SMMUv3 driver reserves any
> > >>>>      RMR region associated with the device such that there is a unity
> > >>>>      mapping for them in SMMU.
> > >>>
> > >>> For the EFI framebuffer use case there is no device to attach so I
> > >>> believe we are left with just the stream ID in bypass mode - which is
> > >>> definitely an improvement (the display works!)
> > >>
> > >> Cool. That’s good to know.
> > >>
> > >>   but not actually a unity
> > >>> mapping of the RMR range. I'm not sure whether it's worth fixing this or
> > >>> not, but I just wanted to point out there's still a need for a driver
> > >>> for the device before the bypass mode is replaced with the unity
> > >>> mapping.
> > >>
> > >> I am not sure either. My idea was we will have bypass STE setup for
> > >> all devices
> > >> with RMR initially and when the corresponding driver takes over(if
> > >> that happens)
> > >> we will have the unity mapping setup properly for the RMR regions. And
> > >> for cases
> > >> like the above, it will remain in the bypass mode.
> > >>
> > >> Do you see any problem(security?) if the dev streams remain in bypass
> > >> mode for
> > >> this dev? Or is it possible to have a stub driver for this dev, so
> > >> that we will have
> > >> the probe/attach invoked and everything will fall in place?
> > >
> > > The downside is that if a driver never binds to that device, it remains
> > > bypassed. If some other externally-controlled malicious device could
> > > manage to spoof that device's requester ID, that could potentially be
> > > exploited.
> > >
> > >> TBH, I haven't looked into creating a temp domain for these types of
> > >> the devices
> > >> and also not sure how we benefit from that compared to the STE bypass
> > >> mode.
> > >
> > > That said, setting up temporary translation contexts that ensure any
> > > access can *only* be to RMR regions until a driver takes over is an
> > > awful lot more work. I'm inclined to be pragmatic here and say we should
> > > get things working at all with the simple bypass STE/S2CR method, then
> > > look at adding the higher-security effort on top.
> > >
> > > Right now systems that need this are either broken (but effectively
> > > secure) or using default bypass with SMMUv2. People who prefer to
> > > maintain security over functionality in the interim can maintain that
> > > status quo by simply continuing to not describe any RMRs.
> >
> > I agree with Robin, let's get this working with the bypass mode (until
> > the device binds) like you've currently got. It's much better than what
> > we have otherwise. Then once that is merged we can look at the temporary
> > translation context or stub driver. The temporary translation context
> > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > and for that it might be possible to do something simpler - if indeed
> > anything is needed at all. I'm not sure how much we need to be worried
> > about malicious devices spoofing requester IDs.
>
> Perfect. I will keep the STE bypass and respin the series once the update
> to the IORT rev E is made public(hope that will happen soon). In the
> meantime, appreciate any feedback on the rest of the patches in this series.

Shameer,

I am pretty sure rev E is already public.  You can find it here.

https://developer.arm.com/documentation/den0049/latest/

It is also marked non-confidential.

I also have initial patches for edk2 and the HoneyComb LX2160a
ACPI tables adding RMR nodes that partially work with your patches.
This is with basic SMMUv2 support but since you have more experience
this this I am more than happy to work with you on your patchset.

-Jon


>
> Thanks,
> Shameer
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 17 December 2020 14:48
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm <linuxarm@huawei.com>;
> Jonathan Cameron <jonathan.cameron@huawei.com>;
> Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Steven Price [mailto:steven.price@arm.com]
> > > Sent: 14 December 2020 13:43
> > > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com>;
> > > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > >
> > > On 14/12/2020 12:33, Robin Murphy wrote:
> > > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > > >> Hi Steve,
> > > >>
> > > >>> -----Original Message-----
> > > >>> From: Steven Price [mailto:steven.price@arm.com]
> > > >>> Sent: 10 December 2020 10:26
> > > >>> To: Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com>;
> > > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > > >>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > >>>
> > > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > > >>>> RFC v1 --> v2:
> > > >>>>    - Added a generic interface for IOMMU drivers to retrieve all the
> > > >>>>      RMR info associated with a given IOMMU.
> > > >>>>    - SMMUv3 driver gets the RMR list during probe() and installs
> > > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
> > > >>>>      based on the suggestions received for v1 to take care of the
> > > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > > >>>
> > > >>> Hi Shameer,
> > > >>>
> > > >>> Sorry for not looking at this before.
> > > >>>
> > > >>> Do you have any plans to implement support in the SMMUv2 driver?
> The
> > > >>> platform I've been testing the EFI framebuffer support on has the
> > > >>> display controller behind SMMUv2, so as it stands this series doesn't
> > > >>> work. I did hack something up for SMMUv2 so I was able to test the
> first
> > > >>> 4 patches.
> > > >>
> > > >> Thanks for taking a look. Sure, I can look into adding the support for
> > > >> SMMUv2.
> > >
> > > Great, thanks!
> > >
> > > >>>
> > > >>>>    - During the probe/attach device, SMMUv3 driver reserves any
> > > >>>>      RMR region associated with the device such that there is a
> unity
> > > >>>>      mapping for them in SMMU.
> > > >>>
> > > >>> For the EFI framebuffer use case there is no device to attach so I
> > > >>> believe we are left with just the stream ID in bypass mode - which is
> > > >>> definitely an improvement (the display works!)
> > > >>
> > > >> Cool. That’s good to know.
> > > >>
> > > >>   but not actually a unity
> > > >>> mapping of the RMR range. I'm not sure whether it's worth fixing this
> or
> > > >>> not, but I just wanted to point out there's still a need for a driver
> > > >>> for the device before the bypass mode is replaced with the unity
> > > >>> mapping.
> > > >>
> > > >> I am not sure either. My idea was we will have bypass STE setup for
> > > >> all devices
> > > >> with RMR initially and when the corresponding driver takes over(if
> > > >> that happens)
> > > >> we will have the unity mapping setup properly for the RMR regions. And
> > > >> for cases
> > > >> like the above, it will remain in the bypass mode.
> > > >>
> > > >> Do you see any problem(security?) if the dev streams remain in bypass
> > > >> mode for
> > > >> this dev? Or is it possible to have a stub driver for this dev, so
> > > >> that we will have
> > > >> the probe/attach invoked and everything will fall in place?
> > > >
> > > > The downside is that if a driver never binds to that device, it remains
> > > > bypassed. If some other externally-controlled malicious device could
> > > > manage to spoof that device's requester ID, that could potentially be
> > > > exploited.
> > > >
> > > >> TBH, I haven't looked into creating a temp domain for these types of
> > > >> the devices
> > > >> and also not sure how we benefit from that compared to the STE bypass
> > > >> mode.
> > > >
> > > > That said, setting up temporary translation contexts that ensure any
> > > > access can *only* be to RMR regions until a driver takes over is an
> > > > awful lot more work. I'm inclined to be pragmatic here and say we should
> > > > get things working at all with the simple bypass STE/S2CR method, then
> > > > look at adding the higher-security effort on top.
> > > >
> > > > Right now systems that need this are either broken (but effectively
> > > > secure) or using default bypass with SMMUv2. People who prefer to
> > > > maintain security over functionality in the interim can maintain that
> > > > status quo by simply continuing to not describe any RMRs.
> > >
> > > I agree with Robin, let's get this working with the bypass mode (until
> > > the device binds) like you've currently got. It's much better than what
> > > we have otherwise. Then once that is merged we can look at the temporary
> > > translation context or stub driver. The temporary translation context
> > > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > > and for that it might be possible to do something simpler - if indeed
> > > anything is needed at all. I'm not sure how much we need to be worried
> > > about malicious devices spoofing requester IDs.
> >
> > Perfect. I will keep the STE bypass and respin the series once the update
> > to the IORT rev E is made public(hope that will happen soon). In the
> > meantime, appreciate any feedback on the rest of the patches in this series.
> 
> Shameer,

Hi Jon,

> 
> I am pretty sure rev E is already public.  You can find it here.
> 
> https://developer.arm.com/documentation/den0049/latest/
> 
> It is also marked non-confidential.

Yes, Rev E is already out there. But I am told that ARM folks are working on
some updates to the IORT spec, especially around the RMR topic. Hopefully
it will be out soon.
 
> 
> I also have initial patches for edk2 and the HoneyComb LX2160a
> ACPI tables adding RMR nodes that partially work with your patches.

Thanks for the update and good to know that it is useful.

Shameer

> This is with basic SMMUv2 support but since you have more experience
> this this I am more than happy to work with you on your patchset.
> 
> -Jon
> 
> 
> >
> > Thanks,
> > Shameer
> > _______________________________________________
> > linux-arm-kernel mailing list
> > linux-arm-kernel@lists.infradead.org
> > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Jon Nettleton]

On Thu, Dec 17, 2020 at 4:42 PM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 17 December 2020 14:48
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> > <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> > linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> > devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm <linuxarm@huawei.com>;
> > Jonathan Cameron <jonathan.cameron@huawei.com>;
> > Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> >
> > On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Steven Price [mailto:steven.price@arm.com]
> > > > Sent: 14 December 2020 13:43
> > > > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum Thodi
> > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > > > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > >
> > > > On 14/12/2020 12:33, Robin Murphy wrote:
> > > > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > > > >> Hi Steve,
> > > > >>
> > > > >>> -----Original Message-----
> > > > >>> From: Steven Price [mailto:steven.price@arm.com]
> > > > >>> Sent: 10 December 2020 10:26
> > > > >>> To: Shameerali Kolothum Thodi
> > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > > > >>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > > > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > > > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > > >>>
> > > > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > > > >>>> RFC v1 --> v2:
> > > > >>>>    - Added a generic interface for IOMMU drivers to retrieve all the
> > > > >>>>      RMR info associated with a given IOMMU.
> > > > >>>>    - SMMUv3 driver gets the RMR list during probe() and installs
> > > > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > > > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
> > > > >>>>      based on the suggestions received for v1 to take care of the
> > > > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > > > >>>
> > > > >>> Hi Shameer,
> > > > >>>
> > > > >>> Sorry for not looking at this before.
> > > > >>>
> > > > >>> Do you have any plans to implement support in the SMMUv2 driver?
> > The
> > > > >>> platform I've been testing the EFI framebuffer support on has the
> > > > >>> display controller behind SMMUv2, so as it stands this series doesn't
> > > > >>> work. I did hack something up for SMMUv2 so I was able to test the
> > first
> > > > >>> 4 patches.
> > > > >>
> > > > >> Thanks for taking a look. Sure, I can look into adding the support for
> > > > >> SMMUv2.
> > > >
> > > > Great, thanks!
> > > >
> > > > >>>
> > > > >>>>    - During the probe/attach device, SMMUv3 driver reserves any
> > > > >>>>      RMR region associated with the device such that there is a
> > unity
> > > > >>>>      mapping for them in SMMU.
> > > > >>>
> > > > >>> For the EFI framebuffer use case there is no device to attach so I
> > > > >>> believe we are left with just the stream ID in bypass mode - which is
> > > > >>> definitely an improvement (the display works!)
> > > > >>
> > > > >> Cool. That’s good to know.
> > > > >>
> > > > >>   but not actually a unity
> > > > >>> mapping of the RMR range. I'm not sure whether it's worth fixing this
> > or
> > > > >>> not, but I just wanted to point out there's still a need for a driver
> > > > >>> for the device before the bypass mode is replaced with the unity
> > > > >>> mapping.
> > > > >>
> > > > >> I am not sure either. My idea was we will have bypass STE setup for
> > > > >> all devices
> > > > >> with RMR initially and when the corresponding driver takes over(if
> > > > >> that happens)
> > > > >> we will have the unity mapping setup properly for the RMR regions. And
> > > > >> for cases
> > > > >> like the above, it will remain in the bypass mode.
> > > > >>
> > > > >> Do you see any problem(security?) if the dev streams remain in bypass
> > > > >> mode for
> > > > >> this dev? Or is it possible to have a stub driver for this dev, so
> > > > >> that we will have
> > > > >> the probe/attach invoked and everything will fall in place?
> > > > >
> > > > > The downside is that if a driver never binds to that device, it remains
> > > > > bypassed. If some other externally-controlled malicious device could
> > > > > manage to spoof that device's requester ID, that could potentially be
> > > > > exploited.
> > > > >
> > > > >> TBH, I haven't looked into creating a temp domain for these types of
> > > > >> the devices
> > > > >> and also not sure how we benefit from that compared to the STE bypass
> > > > >> mode.
> > > > >
> > > > > That said, setting up temporary translation contexts that ensure any
> > > > > access can *only* be to RMR regions until a driver takes over is an
> > > > > awful lot more work. I'm inclined to be pragmatic here and say we should
> > > > > get things working at all with the simple bypass STE/S2CR method, then
> > > > > look at adding the higher-security effort on top.
> > > > >
> > > > > Right now systems that need this are either broken (but effectively
> > > > > secure) or using default bypass with SMMUv2. People who prefer to
> > > > > maintain security over functionality in the interim can maintain that
> > > > > status quo by simply continuing to not describe any RMRs.
> > > >
> > > > I agree with Robin, let's get this working with the bypass mode (until
> > > > the device binds) like you've currently got. It's much better than what
> > > > we have otherwise. Then once that is merged we can look at the temporary
> > > > translation context or stub driver. The temporary translation context
> > > > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > > > and for that it might be possible to do something simpler - if indeed
> > > > anything is needed at all. I'm not sure how much we need to be worried
> > > > about malicious devices spoofing requester IDs.
> > >
> > > Perfect. I will keep the STE bypass and respin the series once the update
> > > to the IORT rev E is made public(hope that will happen soon). In the
> > > meantime, appreciate any feedback on the rest of the patches in this series.
> >
> > Shameer,
>
> Hi Jon,
>
> >
> > I am pretty sure rev E is already public.  You can find it here.
> >
> > https://developer.arm.com/documentation/den0049/latest/
> >
> > It is also marked non-confidential.
>
> Yes, Rev E is already out there. But I am told that ARM folks are working on
> some updates to the IORT spec, especially around the RMR topic. Hopefully
> it will be out soon.

Yes there are some changes coming to the SPEC but I don't know if it is
worth holding up your patchset as an initial implementation.  If you would
like I am more than happy to bring this up as a topic for the next Steering
Committee meeting.

Jon

>
> >
> > I also have initial patches for edk2 and the HoneyComb LX2160a
> > ACPI tables adding RMR nodes that partially work with your patches.
>
> Thanks for the update and good to know that it is useful.
>
> Shameer
>
> > This is with basic SMMUv2 support but since you have more experience
> > this this I am more than happy to work with you on your patchset.
> >
> > -Jon
> >
> >
> > >
> > > Thanks,
> > > Shameer
> > > _______________________________________________
> > > linux-arm-kernel mailing list
> > > linux-arm-kernel@lists.infradead.org
> > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Jon Nettleton]

On Thu, Dec 17, 2020 at 4:53 PM Jon Nettleton <jon@solid-run.com> wrote:
>
> On Thu, Dec 17, 2020 at 4:42 PM Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > Sent: 17 December 2020 14:48
> > > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > > Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> > > <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> > > linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> > > devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org; Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm <linuxarm@huawei.com>;
> > > Jonathan Cameron <jonathan.cameron@huawei.com>;
> > > Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > >
> > > On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Steven Price [mailto:steven.price@arm.com]
> > > > > Sent: 14 December 2020 13:43
> > > > > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum Thodi
> > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>; Guohanjun
> > > > > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > > > > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > > >
> > > > > On 14/12/2020 12:33, Robin Murphy wrote:
> > > > > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > > > > >> Hi Steve,
> > > > > >>
> > > > > >>> -----Original Message-----
> > > > > >>> From: Steven Price [mailto:steven.price@arm.com]
> > > > > >>> Sent: 10 December 2020 10:26
> > > > > >>> To: Shameerali Kolothum Thodi
> > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > >>> Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > > > > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > > > > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > > > >>>
> > > > > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > > > > >>>> RFC v1 --> v2:
> > > > > >>>>    - Added a generic interface for IOMMU drivers to retrieve all the
> > > > > >>>>      RMR info associated with a given IOMMU.
> > > > > >>>>    - SMMUv3 driver gets the RMR list during probe() and installs
> > > > > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > > > > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This is
> > > > > >>>>      based on the suggestions received for v1 to take care of the
> > > > > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > > > > >>>
> > > > > >>> Hi Shameer,
> > > > > >>>
> > > > > >>> Sorry for not looking at this before.
> > > > > >>>
> > > > > >>> Do you have any plans to implement support in the SMMUv2 driver?
> > > The
> > > > > >>> platform I've been testing the EFI framebuffer support on has the
> > > > > >>> display controller behind SMMUv2, so as it stands this series doesn't
> > > > > >>> work. I did hack something up for SMMUv2 so I was able to test the
> > > first
> > > > > >>> 4 patches.
> > > > > >>
> > > > > >> Thanks for taking a look. Sure, I can look into adding the support for
> > > > > >> SMMUv2.
> > > > >
> > > > > Great, thanks!
> > > > >
> > > > > >>>
> > > > > >>>>    - During the probe/attach device, SMMUv3 driver reserves any
> > > > > >>>>      RMR region associated with the device such that there is a
> > > unity
> > > > > >>>>      mapping for them in SMMU.
> > > > > >>>
> > > > > >>> For the EFI framebuffer use case there is no device to attach so I
> > > > > >>> believe we are left with just the stream ID in bypass mode - which is
> > > > > >>> definitely an improvement (the display works!)
> > > > > >>
> > > > > >> Cool. That’s good to know.
> > > > > >>
> > > > > >>   but not actually a unity
> > > > > >>> mapping of the RMR range. I'm not sure whether it's worth fixing this
> > > or
> > > > > >>> not, but I just wanted to point out there's still a need for a driver
> > > > > >>> for the device before the bypass mode is replaced with the unity
> > > > > >>> mapping.
> > > > > >>
> > > > > >> I am not sure either. My idea was we will have bypass STE setup for
> > > > > >> all devices
> > > > > >> with RMR initially and when the corresponding driver takes over(if
> > > > > >> that happens)
> > > > > >> we will have the unity mapping setup properly for the RMR regions. And
> > > > > >> for cases
> > > > > >> like the above, it will remain in the bypass mode.
> > > > > >>
> > > > > >> Do you see any problem(security?) if the dev streams remain in bypass
> > > > > >> mode for
> > > > > >> this dev? Or is it possible to have a stub driver for this dev, so
> > > > > >> that we will have
> > > > > >> the probe/attach invoked and everything will fall in place?
> > > > > >
> > > > > > The downside is that if a driver never binds to that device, it remains
> > > > > > bypassed. If some other externally-controlled malicious device could
> > > > > > manage to spoof that device's requester ID, that could potentially be
> > > > > > exploited.
> > > > > >
> > > > > >> TBH, I haven't looked into creating a temp domain for these types of
> > > > > >> the devices
> > > > > >> and also not sure how we benefit from that compared to the STE bypass
> > > > > >> mode.
> > > > > >
> > > > > > That said, setting up temporary translation contexts that ensure any
> > > > > > access can *only* be to RMR regions until a driver takes over is an
> > > > > > awful lot more work. I'm inclined to be pragmatic here and say we should
> > > > > > get things working at all with the simple bypass STE/S2CR method, then
> > > > > > look at adding the higher-security effort on top.
> > > > > >
> > > > > > Right now systems that need this are either broken (but effectively
> > > > > > secure) or using default bypass with SMMUv2. People who prefer to
> > > > > > maintain security over functionality in the interim can maintain that
> > > > > > status quo by simply continuing to not describe any RMRs.
> > > > >
> > > > > I agree with Robin, let's get this working with the bypass mode (until
> > > > > the device binds) like you've currently got. It's much better than what
> > > > > we have otherwise. Then once that is merged we can look at the temporary
> > > > > translation context or stub driver. The temporary translation context
> > > > > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > > > > and for that it might be possible to do something simpler - if indeed
> > > > > anything is needed at all. I'm not sure how much we need to be worried
> > > > > about malicious devices spoofing requester IDs.
> > > >
> > > > Perfect. I will keep the STE bypass and respin the series once the update
> > > > to the IORT rev E is made public(hope that will happen soon). In the
> > > > meantime, appreciate any feedback on the rest of the patches in this series.
> > >
> > > Shameer,
> >
> > Hi Jon,
> >
> > >
> > > I am pretty sure rev E is already public.  You can find it here.
> > >
> > > https://developer.arm.com/documentation/den0049/latest/
> > >
> > > It is also marked non-confidential.
> >
> > Yes, Rev E is already out there. But I am told that ARM folks are working on
> > some updates to the IORT spec, especially around the RMR topic. Hopefully
> > it will be out soon.
>
> Yes there are some changes coming to the SPEC but I don't know if it is
> worth holding up your patchset as an initial implementation.  If you would
> like I am more than happy to bring this up as a topic for the next Steering
> Committee meeting.
>
> Jon

Shameer,

My first attempt at smmuv2 support can be found in this kernel branch.

https://github.com/SolidRun/linux-stable/commits/linux-5.10.y-cex7

It is functioning if the bypass SMRs are setup in the firmware and RMR's
are exposed in the ACPI tables.  Different from your situation we do want
the device to reclaim the RMR's associated with it on initialization, and I
am still seeing issues there.  I need to spend more time figuring out why
this is not working properly.

-Jon

>
> >
> > >
> > > I also have initial patches for edk2 and the HoneyComb LX2160a
> > > ACPI tables adding RMR nodes that partially work with your patches.
> >
> > Thanks for the update and good to know that it is useful.
> >
> > Shameer
> >
> > > This is with basic SMMUv2 support but since you have more experience
> > > this this I am more than happy to work with you on your patchset.
> > >
> > > -Jon
> > >
> > >
> > > >
> > > > Thanks,
> > > > Shameer
> > > > _______________________________________________
> > > > linux-arm-kernel mailing list
> > > > linux-arm-kernel@lists.infradead.org
> > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 18 December 2020 10:53
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm <linuxarm@huawei.com>;
> Jonathan Cameron <jonathan.cameron@huawei.com>;
> Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> On Thu, Dec 17, 2020 at 4:53 PM Jon Nettleton <jon@solid-run.com> wrote:
> >
> > On Thu, Dec 17, 2020 at 4:42 PM Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com> wrote:
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > Sent: 17 December 2020 14:48
> > > > To: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> > > > Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> > > > <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> > > > linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> > > > devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org;
> Guohanjun
> > > > (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm
> <linuxarm@huawei.com>;
> > > > Jonathan Cameron <jonathan.cameron@huawei.com>;
> > > > Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > >
> > > > On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
> > > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Steven Price [mailto:steven.price@arm.com]
> > > > > > Sent: 14 December 2020 13:43
> > > > > > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum
> Thodi
> > > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > > > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>;
> Guohanjun
> > > > > > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > > > > > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR
> node
> > > > > >
> > > > > > On 14/12/2020 12:33, Robin Murphy wrote:
> > > > > > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > > > > > >> Hi Steve,
> > > > > > >>
> > > > > > >>> -----Original Message-----
> > > > > > >>> From: Steven Price [mailto:steven.price@arm.com]
> > > > > > >>> Sent: 10 December 2020 10:26
> > > > > > >>> To: Shameerali Kolothum Thodi
> > > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > > >>> Cc: Linuxarm <linuxarm@huawei.com>;
> lorenzo.pieralisi@arm.com;
> > > > > > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > > > > > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > > > > > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR
> node
> > > > > > >>>
> > > > > > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > > > > > >>>> RFC v1 --> v2:
> > > > > > >>>>    - Added a generic interface for IOMMU drivers to retrieve all
> the
> > > > > > >>>>      RMR info associated with a given IOMMU.
> > > > > > >>>>    - SMMUv3 driver gets the RMR list during probe() and
> installs
> > > > > > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > > > > > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This
> is
> > > > > > >>>>      based on the suggestions received for v1 to take care of
> the
> > > > > > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > > > > > >>>
> > > > > > >>> Hi Shameer,
> > > > > > >>>
> > > > > > >>> Sorry for not looking at this before.
> > > > > > >>>
> > > > > > >>> Do you have any plans to implement support in the SMMUv2
> driver?
> > > > The
> > > > > > >>> platform I've been testing the EFI framebuffer support on has the
> > > > > > >>> display controller behind SMMUv2, so as it stands this series
> doesn't
> > > > > > >>> work. I did hack something up for SMMUv2 so I was able to test
> the
> > > > first
> > > > > > >>> 4 patches.
> > > > > > >>
> > > > > > >> Thanks for taking a look. Sure, I can look into adding the support for
> > > > > > >> SMMUv2.
> > > > > >
> > > > > > Great, thanks!
> > > > > >
> > > > > > >>>
> > > > > > >>>>    - During the probe/attach device, SMMUv3 driver reserves
> any
> > > > > > >>>>      RMR region associated with the device such that there is
> a
> > > > unity
> > > > > > >>>>      mapping for them in SMMU.
> > > > > > >>>
> > > > > > >>> For the EFI framebuffer use case there is no device to attach so I
> > > > > > >>> believe we are left with just the stream ID in bypass mode - which
> is
> > > > > > >>> definitely an improvement (the display works!)
> > > > > > >>
> > > > > > >> Cool. That’s good to know.
> > > > > > >>
> > > > > > >>   but not actually a unity
> > > > > > >>> mapping of the RMR range. I'm not sure whether it's worth fixing
> this
> > > > or
> > > > > > >>> not, but I just wanted to point out there's still a need for a driver
> > > > > > >>> for the device before the bypass mode is replaced with the unity
> > > > > > >>> mapping.
> > > > > > >>
> > > > > > >> I am not sure either. My idea was we will have bypass STE setup for
> > > > > > >> all devices
> > > > > > >> with RMR initially and when the corresponding driver takes over(if
> > > > > > >> that happens)
> > > > > > >> we will have the unity mapping setup properly for the RMR regions.
> And
> > > > > > >> for cases
> > > > > > >> like the above, it will remain in the bypass mode.
> > > > > > >>
> > > > > > >> Do you see any problem(security?) if the dev streams remain in
> bypass
> > > > > > >> mode for
> > > > > > >> this dev? Or is it possible to have a stub driver for this dev, so
> > > > > > >> that we will have
> > > > > > >> the probe/attach invoked and everything will fall in place?
> > > > > > >
> > > > > > > The downside is that if a driver never binds to that device, it remains
> > > > > > > bypassed. If some other externally-controlled malicious device could
> > > > > > > manage to spoof that device's requester ID, that could potentially be
> > > > > > > exploited.
> > > > > > >
> > > > > > >> TBH, I haven't looked into creating a temp domain for these types
> of
> > > > > > >> the devices
> > > > > > >> and also not sure how we benefit from that compared to the STE
> bypass
> > > > > > >> mode.
> > > > > > >
> > > > > > > That said, setting up temporary translation contexts that ensure any
> > > > > > > access can *only* be to RMR regions until a driver takes over is an
> > > > > > > awful lot more work. I'm inclined to be pragmatic here and say we
> should
> > > > > > > get things working at all with the simple bypass STE/S2CR method,
> then
> > > > > > > look at adding the higher-security effort on top.
> > > > > > >
> > > > > > > Right now systems that need this are either broken (but effectively
> > > > > > > secure) or using default bypass with SMMUv2. People who prefer to
> > > > > > > maintain security over functionality in the interim can maintain that
> > > > > > > status quo by simply continuing to not describe any RMRs.
> > > > > >
> > > > > > I agree with Robin, let's get this working with the bypass mode (until
> > > > > > the device binds) like you've currently got. It's much better than what
> > > > > > we have otherwise. Then once that is merged we can look at the
> temporary
> > > > > > translation context or stub driver. The temporary translation context
> > > > > > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > > > > > and for that it might be possible to do something simpler - if indeed
> > > > > > anything is needed at all. I'm not sure how much we need to be
> worried
> > > > > > about malicious devices spoofing requester IDs.
> > > > >
> > > > > Perfect. I will keep the STE bypass and respin the series once the update
> > > > > to the IORT rev E is made public(hope that will happen soon). In the
> > > > > meantime, appreciate any feedback on the rest of the patches in this
> series.
> > > >
> > > > Shameer,
> > >
> > > Hi Jon,
> > >
> > > >
> > > > I am pretty sure rev E is already public.  You can find it here.
> > > >
> > > > https://developer.arm.com/documentation/den0049/latest/
> > > >
> > > > It is also marked non-confidential.
> > >
> > > Yes, Rev E is already out there. But I am told that ARM folks are working on
> > > some updates to the IORT spec, especially around the RMR topic. Hopefully
> > > it will be out soon.
> >
> > Yes there are some changes coming to the SPEC but I don't know if it is
> > worth holding up your patchset as an initial implementation.  If you would
> > like I am more than happy to bring this up as a topic for the next Steering
> > Committee meeting.
> >
> > Jon
>
> Shameer,
>

Hi Jon,
 
> My first attempt at smmuv2 support can be found in this kernel branch.
> 
> https://github.com/SolidRun/linux-stable/commits/linux-5.10.y-cex7
> 
> It is functioning if the bypass SMRs are setup in the firmware and RMR's
> are exposed in the ACPI tables.

Ok. Thanks for sharing it. Happy to carry those patches as part of this
series if you are fine with it.

  Different from your situation we do want
> the device to reclaim the RMR's associated with it on initialization, and I
> am still seeing issues there.  I need to spend more time figuring out why
> this is not working properly.

I am not sure what your requirement is here. So if the kernel driver eventually
comes up and takes control of the device, you no longer require the bypass/identity
mapping for these regions. Is that correct?

Shameer

> -Jon
> 
> >
> > >
> > > >
> > > > I also have initial patches for edk2 and the HoneyComb LX2160a
> > > > ACPI tables adding RMR nodes that partially work with your patches.
> > >
> > > Thanks for the update and good to know that it is useful.
> > >
> > > Shameer
> > >
> > > > This is with basic SMMUv2 support but since you have more experience
> > > > this this I am more than happy to work with you on your patchset.
> > > >
> > > > -Jon
> > > >
> > > >
> > > > >
> > > > > Thanks,
> > > > > Shameer
> > > > > _______________________________________________
> > > > > linux-arm-kernel mailing list
> > > > > linux-arm-kernel@lists.infradead.org
> > > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Jon Nettleton]

On Mon, Jan 4, 2021 at 9:55 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
>
>
> > -----Original Message-----
> > From: Jon Nettleton [mailto:jon@solid-run.com]
> > Sent: 18 December 2020 10:53
> > To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> > Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> > <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> > linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> > devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm <linuxarm@huawei.com>;
> > Jonathan Cameron <jonathan.cameron@huawei.com>;
> > Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> >
> > On Thu, Dec 17, 2020 at 4:53 PM Jon Nettleton <jon@solid-run.com> wrote:
> > >
> > > On Thu, Dec 17, 2020 at 4:42 PM Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: Jon Nettleton [mailto:jon@solid-run.com]
> > > > > Sent: 17 December 2020 14:48
> > > > > To: Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>
> > > > > Cc: Steven Price <steven.price@arm.com>; Robin Murphy
> > > > > <robin.murphy@arm.com>; linux-arm-kernel@lists.infradead.org;
> > > > > linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> > > > > devel@acpica.org; lorenzo.pieralisi@arm.com; joro@8bytes.org;
> > Guohanjun
> > > > > (Hanjun Guo) <guohanjun@huawei.com>; Linuxarm
> > <linuxarm@huawei.com>;
> > > > > Jonathan Cameron <jonathan.cameron@huawei.com>;
> > > > > Sami.Mujawar@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> > > > >
> > > > > On Mon, Dec 14, 2020 at 3:48 PM Shameerali Kolothum Thodi
> > > > > <shameerali.kolothum.thodi@huawei.com> wrote:
> > > > > >
> > > > > >
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Steven Price [mailto:steven.price@arm.com]
> > > > > > > Sent: 14 December 2020 13:43
> > > > > > > To: Robin Murphy <robin.murphy@arm.com>; Shameerali Kolothum
> > Thodi
> > > > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > > > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > > > Cc: Linuxarm <linuxarm@huawei.com>; lorenzo.pieralisi@arm.com;
> > > > > > > joro@8bytes.org; wanghuiqiang <wanghuiqiang@huawei.com>;
> > Guohanjun
> > > > > > > (Hanjun Guo) <guohanjun@huawei.com>; Jonathan Cameron
> > > > > > > <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > > > Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR
> > node
> > > > > > >
> > > > > > > On 14/12/2020 12:33, Robin Murphy wrote:
> > > > > > > > On 2020-12-14 10:55, Shameerali Kolothum Thodi wrote:
> > > > > > > >> Hi Steve,
> > > > > > > >>
> > > > > > > >>> -----Original Message-----
> > > > > > > >>> From: Steven Price [mailto:steven.price@arm.com]
> > > > > > > >>> Sent: 10 December 2020 10:26
> > > > > > > >>> To: Shameerali Kolothum Thodi
> > > > > > > <shameerali.kolothum.thodi@huawei.com>;
> > > > > > > >>> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > > > >>> iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > > > >>> Cc: Linuxarm <linuxarm@huawei.com>;
> > lorenzo.pieralisi@arm.com;
> > > > > > > >>> joro@8bytes.org; robin.murphy@arm.com; wanghuiqiang
> > > > > > > >>> <wanghuiqiang@huawei.com>; Guohanjun (Hanjun Guo)
> > > > > > > >>> <guohanjun@huawei.com>; Jonathan Cameron
> > > > > > > >>> <jonathan.cameron@huawei.com>; Sami.Mujawar@arm.com
> > > > > > > >>> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR
> > node
> > > > > > > >>>
> > > > > > > >>> On 19/11/2020 12:11, Shameer Kolothum wrote:
> > > > > > > >>>> RFC v1 --> v2:
> > > > > > > >>>>    - Added a generic interface for IOMMU drivers to retrieve all
> > the
> > > > > > > >>>>      RMR info associated with a given IOMMU.
> > > > > > > >>>>    - SMMUv3 driver gets the RMR list during probe() and
> > installs
> > > > > > > >>>>      bypass STEs for all the SIDs in the RMR list. This is to keep
> > > > > > > >>>>      the ongoing traffic alive(if any) during SMMUv3 reset. This
> > is
> > > > > > > >>>>      based on the suggestions received for v1 to take care of
> > the
> > > > > > > >>>>      EFI framebuffer use case. Only sanity tested for now.
> > > > > > > >>>
> > > > > > > >>> Hi Shameer,
> > > > > > > >>>
> > > > > > > >>> Sorry for not looking at this before.
> > > > > > > >>>
> > > > > > > >>> Do you have any plans to implement support in the SMMUv2
> > driver?
> > > > > The
> > > > > > > >>> platform I've been testing the EFI framebuffer support on has the
> > > > > > > >>> display controller behind SMMUv2, so as it stands this series
> > doesn't
> > > > > > > >>> work. I did hack something up for SMMUv2 so I was able to test
> > the
> > > > > first
> > > > > > > >>> 4 patches.
> > > > > > > >>
> > > > > > > >> Thanks for taking a look. Sure, I can look into adding the support for
> > > > > > > >> SMMUv2.
> > > > > > >
> > > > > > > Great, thanks!
> > > > > > >
> > > > > > > >>>
> > > > > > > >>>>    - During the probe/attach device, SMMUv3 driver reserves
> > any
> > > > > > > >>>>      RMR region associated with the device such that there is
> > a
> > > > > unity
> > > > > > > >>>>      mapping for them in SMMU.
> > > > > > > >>>
> > > > > > > >>> For the EFI framebuffer use case there is no device to attach so I
> > > > > > > >>> believe we are left with just the stream ID in bypass mode - which
> > is
> > > > > > > >>> definitely an improvement (the display works!)
> > > > > > > >>
> > > > > > > >> Cool. That’s good to know.
> > > > > > > >>
> > > > > > > >>   but not actually a unity
> > > > > > > >>> mapping of the RMR range. I'm not sure whether it's worth fixing
> > this
> > > > > or
> > > > > > > >>> not, but I just wanted to point out there's still a need for a driver
> > > > > > > >>> for the device before the bypass mode is replaced with the unity
> > > > > > > >>> mapping.
> > > > > > > >>
> > > > > > > >> I am not sure either. My idea was we will have bypass STE setup for
> > > > > > > >> all devices
> > > > > > > >> with RMR initially and when the corresponding driver takes over(if
> > > > > > > >> that happens)
> > > > > > > >> we will have the unity mapping setup properly for the RMR regions.
> > And
> > > > > > > >> for cases
> > > > > > > >> like the above, it will remain in the bypass mode.
> > > > > > > >>
> > > > > > > >> Do you see any problem(security?) if the dev streams remain in
> > bypass
> > > > > > > >> mode for
> > > > > > > >> this dev? Or is it possible to have a stub driver for this dev, so
> > > > > > > >> that we will have
> > > > > > > >> the probe/attach invoked and everything will fall in place?
> > > > > > > >
> > > > > > > > The downside is that if a driver never binds to that device, it remains
> > > > > > > > bypassed. If some other externally-controlled malicious device could
> > > > > > > > manage to spoof that device's requester ID, that could potentially be
> > > > > > > > exploited.
> > > > > > > >
> > > > > > > >> TBH, I haven't looked into creating a temp domain for these types
> > of
> > > > > > > >> the devices
> > > > > > > >> and also not sure how we benefit from that compared to the STE
> > bypass
> > > > > > > >> mode.
> > > > > > > >
> > > > > > > > That said, setting up temporary translation contexts that ensure any
> > > > > > > > access can *only* be to RMR regions until a driver takes over is an
> > > > > > > > awful lot more work. I'm inclined to be pragmatic here and say we
> > should
> > > > > > > > get things working at all with the simple bypass STE/S2CR method,
> > then
> > > > > > > > look at adding the higher-security effort on top.
> > > > > > > >
> > > > > > > > Right now systems that need this are either broken (but effectively
> > > > > > > > secure) or using default bypass with SMMUv2. People who prefer to
> > > > > > > > maintain security over functionality in the interim can maintain that
> > > > > > > > status quo by simply continuing to not describe any RMRs.
> > > > > > >
> > > > > > > I agree with Robin, let's get this working with the bypass mode (until
> > > > > > > the device binds) like you've currently got. It's much better than what
> > > > > > > we have otherwise. Then once that is merged we can look at the
> > temporary
> > > > > > > translation context or stub driver. The temporary translation context
> > > > > > > would be 'neatest', but I'm only aware of the EFI framebuffer use case
> > > > > > > and for that it might be possible to do something simpler - if indeed
> > > > > > > anything is needed at all. I'm not sure how much we need to be
> > worried
> > > > > > > about malicious devices spoofing requester IDs.
> > > > > >
> > > > > > Perfect. I will keep the STE bypass and respin the series once the update
> > > > > > to the IORT rev E is made public(hope that will happen soon). In the
> > > > > > meantime, appreciate any feedback on the rest of the patches in this
> > series.
> > > > >
> > > > > Shameer,
> > > >
> > > > Hi Jon,
> > > >
> > > > >
> > > > > I am pretty sure rev E is already public.  You can find it here.
> > > > >
> > > > > https://developer.arm.com/documentation/den0049/latest/
> > > > >
> > > > > It is also marked non-confidential.
> > > >
> > > > Yes, Rev E is already out there. But I am told that ARM folks are working on
> > > > some updates to the IORT spec, especially around the RMR topic. Hopefully
> > > > it will be out soon.
> > >
> > > Yes there are some changes coming to the SPEC but I don't know if it is
> > > worth holding up your patchset as an initial implementation.  If you would
> > > like I am more than happy to bring this up as a topic for the next Steering
> > > Committee meeting.
> > >
> > > Jon
> >
> > Shameer,
> >
>
> Hi Jon,
>
> > My first attempt at smmuv2 support can be found in this kernel branch.
> >
> > https://github.com/SolidRun/linux-stable/commits/linux-5.10.y-cex7
> >
> > It is functioning if the bypass SMRs are setup in the firmware and RMR's
> > are exposed in the ACPI tables.
>
> Ok. Thanks for sharing it. Happy to carry those patches as part of this
> series if you are fine with it.

That is fine with me.  I have also cc'd Laurentiu from NXP as part of the
code is lifted from his integration with the Qualcom patches submitted
for device-tree handling.

We may want to also start a conversation if the RMR list should be
populated from the device-tree codepaths and then have the SMMU*
drivers share the same mechanism for adding the bypass mappings
and then letting drivers reclaim the DMA regions.

I think there was an Nvidia patchset that did something similar based
on DT reserved memory regions and smmu mappings.

>
>   Different from your situation we do want
> > the device to reclaim the RMR's associated with it on initialization, and I
> > am still seeing issues there.  I need to spend more time figuring out why
> > this is not working properly.
>
> I am not sure what your requirement is here. So if the kernel driver eventually
> comes up and takes control of the device, you no longer require the bypass/identity
> mapping for these regions. Is that correct?

Yes this is basically correct.  I believe some of the issues I have
run into will
be covered in the next release of the RMR specs.  For now this patchset
does give me basic working functionality, and the future refinements will
add more fine grained security that I am looking for.

As I mentioned before, it would be nice to get this functionality moving
forward and then we can refine it as spec updates are made public.

-Jon

>
> Shameer
>
> > -Jon
> >
> > >
> > > >
> > > > >
> > > > > I also have initial patches for edk2 and the HoneyComb LX2160a
> > > > > ACPI tables adding RMR nodes that partially work with your patches.
> > > >
> > > > Thanks for the update and good to know that it is useful.
> > > >
> > > > Shameer
> > > >
> > > > > This is with basic SMMUv2 support but since you have more experience
> > > > > this this I am more than happy to work with you on your patchset.
> > > > >
> > > > > -Jon
> > > > >
> > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Shameer
> > > > > > _______________________________________________
> > > > > > linux-arm-kernel mailing list
> > > > > > linux-arm-kernel@lists.infradead.org
> > > > > > http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Shameerali Kolothum Thodi]

[+]

Hi Erik,

As this is now just merged ino acpica-master and based on the discussion we had here,

https://github.com/acpica/acpica/pull/638

I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions call and
can confirm that the IORT Revision E is not the final specification and has some issues
which is now corrected in the latest E.b revision[1]. Also there are no current users
for the Rev E and it may not be a good idea to push this version into the Linux kernel
or elsewhere.

So could you please revert the merge and I am planning to work on the E.b soon.
Please let me know if I need to explicitly send a revert pull request or not.

Thanks,
Shameer

1. https://developer.arm.com/documentation/den0049/latest/

> -----Original Message-----
> From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf Of
> Shameer Kolothum
> Sent: 19 November 2020 12:12
> To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> 
> IORT revision E contains a few additions like,
>     -Added an identifier field in the node descriptors to aid table
>      cross-referencing.
>     -Introduced the Reserved Memory Range(RMR) node. This is used
>      to describe memory ranges that are used by endpoints and requires
>      a unity mapping in SMMU.
>     -Introduced a flag in the RC node to express support for PRI.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  include/acpi/actbl2.h | 25 +++++++++++++++++++------
>  1 file changed, 19 insertions(+), 6 deletions(-)
> 
> diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> ec66779cb193..274fce7b5c01 100644
> --- a/include/acpi/actbl2.h
> +++ b/include/acpi/actbl2.h
> @@ -68,7 +68,7 @@
>   * IORT - IO Remapping Table
>   *
>   * Conforms to "IO Remapping Table System Software on ARM Platforms",
> - * Document number: ARM DEN 0049D, March 2018
> + * Document number: ARM DEN 0049E, June 2020
>   *
> 
> ****************************************************************
> **************/
> 
> @@ -86,7 +86,8 @@ struct acpi_iort_node {
>  	u8 type;
>  	u16 length;
>  	u8 revision;
> -	u32 reserved;
> +	u16 reserved;
> +	u16 identifier;
>  	u32 mapping_count;
>  	u32 mapping_offset;
>  	char node_data[1];
> @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
>  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
>  	ACPI_IORT_NODE_SMMU = 0x03,
>  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> -	ACPI_IORT_NODE_PMCG = 0x05
> +	ACPI_IORT_NODE_PMCG = 0x05,
> +	ACPI_IORT_NODE_RMR = 0x06,
>  };
> 
>  struct acpi_iort_id_mapping {
> @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
>  	u8 reserved[3];		/* Reserved, must be zero */
>  };
> 
> -/* Values for ats_attribute field above */
> +/* Masks for ats_attribute field above */
> 
> -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root
> complex supports ATS */
> -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root
> complex doesn't support ATS */
> +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex
> supports ATS */
> +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root complex
> supports PRI */
> 
>  struct acpi_iort_smmu {
>  	u64 base_address;	/* SMMU base address */
> @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
>  	u64 page1_base_address;
>  };
> 
> +struct acpi_iort_rmr {
> +	u32 rmr_count;
> +	u32 rmr_offset;
> +};
> +
> +struct acpi_iort_rmr_desc {
> +	u64 base_address;
> +	u64 length;
> +	u32 reserved;
> +};
> +
> 
> /***************************************************************
> ****************
>   *
>   * IVRS - I/O Virtualization Reporting Structure
> --
> 2.17.1
> 
> _______________________________________________
> iommu mailing list
> iommu@lists.linux-foundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu

RE: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Kaneda, Erik]

> -----Original Message-----
> From: Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> Sent: Monday, March 22, 2021 3:36 AM
> To: Kaneda, Erik <erik.kaneda@intel.com>; linux-arm-
> kernel@lists.infradead.org; linux-acpi@vger.kernel.org; iommu@lists.linux-
> foundation.org; devel@acpica.org; Lorenzo Pieralisi
> <lorenzo.pieralisi@arm.com>; Moore, Robert <robert.moore@intel.com>
> Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> Sami.Mujawar@arm.com; robin.murphy@arm.com; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: [Devel] Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> 
> [+]
> 
> Hi Erik,
> 
> As this is now just merged ino acpica-master and based on the discussion we
> had here,
> 
> https://github.com/acpica/acpica/pull/638
> 
> I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions call
> and
> can confirm that the IORT Revision E is not the final specification and has
> some issues
> which is now corrected in the latest E.b revision[1]. Also there are no current
> users
> for the Rev E and it may not be a good idea to push this version into the Linux
> kernel
> or elsewhere.
> 
> So could you please revert the merge and I am planning to work on the E.b
> soon.
Hi,

> Please let me know if I need to explicitly send a revert pull request or not.

Please send a revert pull request and I'll remember to not submit Linux-ize the IORT patches.

From all of the activity that I've seen with the IORT specification, it looks like this table is nontrivial to design and maintain. The difficulty I have with the table is that I do not understand which table revisions are in active use.

So my question is this: which IORT revisions are being actively used?

Thanks,
Erik
> 
> Thanks,
> Shameer
> 
> 1. https://developer.arm.com/documentation/den0049/latest/
> 
> > -----Original Message-----
> > From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On
> Behalf Of
> > Shameer Kolothum
> > Sent: 19 November 2020 12:12
> > To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-foundation.org; devel@acpica.org
> > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> > robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> >
> > IORT revision E contains a few additions like,
> >     -Added an identifier field in the node descriptors to aid table
> >      cross-referencing.
> >     -Introduced the Reserved Memory Range(RMR) node. This is used
> >      to describe memory ranges that are used by endpoints and requires
> >      a unity mapping in SMMU.
> >     -Introduced a flag in the RC node to express support for PRI.
> >
> > Signed-off-by: Shameer Kolothum
> <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  include/acpi/actbl2.h | 25 +++++++++++++++++++------
> >  1 file changed, 19 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> > ec66779cb193..274fce7b5c01 100644
> > --- a/include/acpi/actbl2.h
> > +++ b/include/acpi/actbl2.h
> > @@ -68,7 +68,7 @@
> >   * IORT - IO Remapping Table
> >   *
> >   * Conforms to "IO Remapping Table System Software on ARM Platforms",
> > - * Document number: ARM DEN 0049D, March 2018
> > + * Document number: ARM DEN 0049E, June 2020
> >   *
> >
> >
> **********************************************************
> ******
> > **************/
> >
> > @@ -86,7 +86,8 @@ struct acpi_iort_node {
> >  	u8 type;
> >  	u16 length;
> >  	u8 revision;
> > -	u32 reserved;
> > +	u16 reserved;
> > +	u16 identifier;
> >  	u32 mapping_count;
> >  	u32 mapping_offset;
> >  	char node_data[1];
> > @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
> >  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
> >  	ACPI_IORT_NODE_SMMU = 0x03,
> >  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> > -	ACPI_IORT_NODE_PMCG = 0x05
> > +	ACPI_IORT_NODE_PMCG = 0x05,
> > +	ACPI_IORT_NODE_RMR = 0x06,
> >  };
> >
> >  struct acpi_iort_id_mapping {
> > @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
> >  	u8 reserved[3];		/* Reserved, must be zero */
> >  };
> >
> > -/* Values for ats_attribute field above */
> > +/* Masks for ats_attribute field above */
> >
> > -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root
> > complex supports ATS */
> > -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root
> > complex doesn't support ATS */
> > +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex
> > supports ATS */
> > +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root complex
> > supports PRI */
> >
> >  struct acpi_iort_smmu {
> >  	u64 base_address;	/* SMMU base address */
> > @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
> >  	u64 page1_base_address;
> >  };
> >
> > +struct acpi_iort_rmr {
> > +	u32 rmr_count;
> > +	u32 rmr_offset;
> > +};
> > +
> > +struct acpi_iort_rmr_desc {
> > +	u64 base_address;
> > +	u64 length;
> > +	u32 reserved;
> > +};
> > +
> >
> >
> /**********************************************************
> *****
> > ****************
> >   *
> >   * IVRS - I/O Virtualization Reporting Structure
> > --
> > 2.17.1
> >
> > _______________________________________________
> > iommu mailing list
> > iommu@lists.linux-foundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> _______________________________________________
> Devel mailing list -- devel@acpica.org
> To unsubscribe send an email to devel-leave@acpica.org
> %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Lorenzo Pieralisi]

On Mon, Mar 22, 2021 at 09:57:58PM +0000, Kaneda, Erik wrote:
> 
> 
> > -----Original Message-----
> > From: Shameerali Kolothum Thodi
> > <shameerali.kolothum.thodi@huawei.com>
> > Sent: Monday, March 22, 2021 3:36 AM
> > To: Kaneda, Erik <erik.kaneda@intel.com>; linux-arm-
> > kernel@lists.infradead.org; linux-acpi@vger.kernel.org; iommu@lists.linux-
> > foundation.org; devel@acpica.org; Lorenzo Pieralisi
> > <lorenzo.pieralisi@arm.com>; Moore, Robert <robert.moore@intel.com>
> > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > Sami.Mujawar@arm.com; robin.murphy@arm.com; wanghuiqiang
> > <wanghuiqiang@huawei.com>
> > Subject: [Devel] Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> > 
> > [+]
> > 
> > Hi Erik,
> > 
> > As this is now just merged ino acpica-master and based on the discussion we
> > had here,
> > 
> > https://github.com/acpica/acpica/pull/638
> > 
> > I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions call
> > and
> > can confirm that the IORT Revision E is not the final specification and has
> > some issues
> > which is now corrected in the latest E.b revision[1]. Also there are no current
> > users
> > for the Rev E and it may not be a good idea to push this version into the Linux
> > kernel
> > or elsewhere.
> > 
> > So could you please revert the merge and I am planning to work on the E.b
> > soon.
> Hi,
> 
> > Please let me know if I need to explicitly send a revert pull request or not.
> 
> Please send a revert pull request and I'll remember to not submit Linux-ize the IORT patches.
> 
> From all of the activity that I've seen with the IORT specification,
> it looks like this table is nontrivial to design and maintain. The
> difficulty I have with the table is that I do not understand which
> table revisions are in active use.

Possibly all of them in firmware in the field - I am not sure what you
are asking though; if you can elaborate I'd be grateful.

> So my question is this: which IORT revisions are being actively used?

See above.

Thanks,
Lorenzo

> 
> Thanks,
> Erik
> > 
> > Thanks,
> > Shameer
> > 
> > 1. https://developer.arm.com/documentation/den0049/latest/
> > 
> > > -----Original Message-----
> > > From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On
> > Behalf Of
> > > Shameer Kolothum
> > > Sent: 19 November 2020 12:12
> > > To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > Guohanjun
> > > (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> > > robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> > >
> > > IORT revision E contains a few additions like,
> > >     -Added an identifier field in the node descriptors to aid table
> > >      cross-referencing.
> > >     -Introduced the Reserved Memory Range(RMR) node. This is used
> > >      to describe memory ranges that are used by endpoints and requires
> > >      a unity mapping in SMMU.
> > >     -Introduced a flag in the RC node to express support for PRI.
> > >
> > > Signed-off-by: Shameer Kolothum
> > <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >  include/acpi/actbl2.h | 25 +++++++++++++++++++------
> > >  1 file changed, 19 insertions(+), 6 deletions(-)
> > >
> > > diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> > > ec66779cb193..274fce7b5c01 100644
> > > --- a/include/acpi/actbl2.h
> > > +++ b/include/acpi/actbl2.h
> > > @@ -68,7 +68,7 @@
> > >   * IORT - IO Remapping Table
> > >   *
> > >   * Conforms to "IO Remapping Table System Software on ARM Platforms",
> > > - * Document number: ARM DEN 0049D, March 2018
> > > + * Document number: ARM DEN 0049E, June 2020
> > >   *
> > >
> > >
> > **********************************************************
> > ******
> > > **************/
> > >
> > > @@ -86,7 +86,8 @@ struct acpi_iort_node {
> > >  	u8 type;
> > >  	u16 length;
> > >  	u8 revision;
> > > -	u32 reserved;
> > > +	u16 reserved;
> > > +	u16 identifier;
> > >  	u32 mapping_count;
> > >  	u32 mapping_offset;
> > >  	char node_data[1];
> > > @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
> > >  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
> > >  	ACPI_IORT_NODE_SMMU = 0x03,
> > >  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> > > -	ACPI_IORT_NODE_PMCG = 0x05
> > > +	ACPI_IORT_NODE_PMCG = 0x05,
> > > +	ACPI_IORT_NODE_RMR = 0x06,
> > >  };
> > >
> > >  struct acpi_iort_id_mapping {
> > > @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
> > >  	u8 reserved[3];		/* Reserved, must be zero */
> > >  };
> > >
> > > -/* Values for ats_attribute field above */
> > > +/* Masks for ats_attribute field above */
> > >
> > > -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root
> > > complex supports ATS */
> > > -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root
> > > complex doesn't support ATS */
> > > +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex
> > > supports ATS */
> > > +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root complex
> > > supports PRI */
> > >
> > >  struct acpi_iort_smmu {
> > >  	u64 base_address;	/* SMMU base address */
> > > @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
> > >  	u64 page1_base_address;
> > >  };
> > >
> > > +struct acpi_iort_rmr {
> > > +	u32 rmr_count;
> > > +	u32 rmr_offset;
> > > +};
> > > +
> > > +struct acpi_iort_rmr_desc {
> > > +	u64 base_address;
> > > +	u64 length;
> > > +	u32 reserved;
> > > +};
> > > +
> > >
> > >
> > /**********************************************************
> > *****
> > > ****************
> > >   *
> > >   * IVRS - I/O Virtualization Reporting Structure
> > > --
> > > 2.17.1
> > >
> > > _______________________________________________
> > > iommu mailing list
> > > iommu@lists.linux-foundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> > _______________________________________________
> > Devel mailing list -- devel@acpica.org
> > To unsubscribe send an email to devel-leave@acpica.org
> > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

RE: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Kaneda, Erik]

> -----Original Message-----
> From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Sent: Tuesday, March 23, 2021 8:54 AM
> To: Kaneda, Erik <erik.kaneda@intel.com>
> Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org; Moore, Robert
> <robert.moore@intel.com>; Linuxarm <linuxarm@huawei.com>;
> steven.price@arm.com; Sami.Mujawar@arm.com; robin.murphy@arm.com;
> wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> 
> On Mon, Mar 22, 2021 at 09:57:58PM +0000, Kaneda, Erik wrote:
> >
> >
> > > -----Original Message-----
> > > From: Shameerali Kolothum Thodi
> > > <shameerali.kolothum.thodi@huawei.com>
> > > Sent: Monday, March 22, 2021 3:36 AM
> > > To: Kaneda, Erik <erik.kaneda@intel.com>; linux-arm-
> > > kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-
> > > foundation.org; devel@acpica.org; Lorenzo Pieralisi
> > > <lorenzo.pieralisi@arm.com>; Moore, Robert
> <robert.moore@intel.com>
> > > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > > Sami.Mujawar@arm.com; robin.murphy@arm.com; wanghuiqiang
> > > <wanghuiqiang@huawei.com>
> > > Subject: [Devel] Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for
> revision E
> > >
> > > [+]
> > >
> > > Hi Erik,
> > >
> > > As this is now just merged ino acpica-master and based on the discussion
> we
> > > had here,
> > >
> > > https://github.com/acpica/acpica/pull/638
> > >
> > > I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions
> call
> > > and
> > > can confirm that the IORT Revision E is not the final specification and has
> > > some issues
> > > which is now corrected in the latest E.b revision[1]. Also there are no
> current
> > > users
> > > for the Rev E and it may not be a good idea to push this version into the
> Linux
> > > kernel
> > > or elsewhere.
> > >
> > > So could you please revert the merge and I am planning to work on the
> E.b
> > > soon.
> > Hi,
> >
> > > Please let me know if I need to explicitly send a revert pull request or not.
> >
> > Please send a revert pull request and I'll remember to not submit Linux-ize
> the IORT patches.
> >
> > From all of the activity that I've seen with the IORT specification,
> > it looks like this table is nontrivial to design and maintain. The
> > difficulty I have with the table is that I do not understand which
> > table revisions are in active use.
> 
Hi Lorenzo,

> Possibly all of them in firmware in the field - I am not sure what you
> are asking though; if you can elaborate I'd be grateful.

Yes, I'd be happy to explain.

The ACPICA project contains code that provides definitions for ACPI tables. After each release of this project, the codebase gets translated to a Linux style syntax and relevant patches are submitted to Linux so that it can use these table definitions in their driver codebase. From ACPICA's perspective, the code providing these definitions are used to implement a compiler/disassembler called iASL. This tool disassembles table definitions so that the user doesn't have to open a hex editor to decipher ACPI tables. Our goal with iASL is to be able to disassemble as many ACPI tables as possible to give users the ability to compile and debug ACPI tables.

Out of the 60+ tables that we support, IORT has been tricky to maintain. There are revisions A-E and we have received pull requests from engineers from ARM or companies that work with ARM. We are grateful of their contributions but some of these pull requests have broken support for earlier versions of IORT. In addition, we sometimes receive communication from people like Shameer saying that revision E does not currently have users. This leaves Bob and I very confused about which revisions we should be focused on supporting in iASL.

We need your help in understanding which versions of IORT should be supported by iASL as well as Linux.

I hope this helps.. Let me know if you need me to clarify anything that I've written.

Thanks,
Erik
> 
> > So my question is this: which IORT revisions are being actively used?
> 
> See above.
> 
> Thanks,
> Lorenzo
> 
> >
> > Thanks,
> > Erik
> > >
> > > Thanks,
> > > Shameer
> > >
> > > 1. https://developer.arm.com/documentation/den0049/latest/
> > >
> > > > -----Original Message-----
> > > > From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On
> > > Behalf Of
> > > > Shameer Kolothum
> > > > Sent: 19 November 2020 12:12
> > > > To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > > Guohanjun
> > > > (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> > > > robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > > Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> > > >
> > > > IORT revision E contains a few additions like,
> > > >     -Added an identifier field in the node descriptors to aid table
> > > >      cross-referencing.
> > > >     -Introduced the Reserved Memory Range(RMR) node. This is used
> > > >      to describe memory ranges that are used by endpoints and requires
> > > >      a unity mapping in SMMU.
> > > >     -Introduced a flag in the RC node to express support for PRI.
> > > >
> > > > Signed-off-by: Shameer Kolothum
> > > <shameerali.kolothum.thodi@huawei.com>
> > > > ---
> > > >  include/acpi/actbl2.h | 25 +++++++++++++++++++------
> > > >  1 file changed, 19 insertions(+), 6 deletions(-)
> > > >
> > > > diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> > > > ec66779cb193..274fce7b5c01 100644
> > > > --- a/include/acpi/actbl2.h
> > > > +++ b/include/acpi/actbl2.h
> > > > @@ -68,7 +68,7 @@
> > > >   * IORT - IO Remapping Table
> > > >   *
> > > >   * Conforms to "IO Remapping Table System Software on ARM
> Platforms",
> > > > - * Document number: ARM DEN 0049D, March 2018
> > > > + * Document number: ARM DEN 0049E, June 2020
> > > >   *
> > > >
> > > >
> > >
> **********************************************************
> > > ******
> > > > **************/
> > > >
> > > > @@ -86,7 +86,8 @@ struct acpi_iort_node {
> > > >  	u8 type;
> > > >  	u16 length;
> > > >  	u8 revision;
> > > > -	u32 reserved;
> > > > +	u16 reserved;
> > > > +	u16 identifier;
> > > >  	u32 mapping_count;
> > > >  	u32 mapping_offset;
> > > >  	char node_data[1];
> > > > @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
> > > >  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
> > > >  	ACPI_IORT_NODE_SMMU = 0x03,
> > > >  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> > > > -	ACPI_IORT_NODE_PMCG = 0x05
> > > > +	ACPI_IORT_NODE_PMCG = 0x05,
> > > > +	ACPI_IORT_NODE_RMR = 0x06,
> > > >  };
> > > >
> > > >  struct acpi_iort_id_mapping {
> > > > @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
> > > >  	u8 reserved[3];		/* Reserved, must be zero */
> > > >  };
> > > >
> > > > -/* Values for ats_attribute field above */
> > > > +/* Masks for ats_attribute field above */
> > > >
> > > > -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root
> > > > complex supports ATS */
> > > > -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root
> > > > complex doesn't support ATS */
> > > > +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex
> > > > supports ATS */
> > > > +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root
> complex
> > > > supports PRI */
> > > >
> > > >  struct acpi_iort_smmu {
> > > >  	u64 base_address;	/* SMMU base address */
> > > > @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
> > > >  	u64 page1_base_address;
> > > >  };
> > > >
> > > > +struct acpi_iort_rmr {
> > > > +	u32 rmr_count;
> > > > +	u32 rmr_offset;
> > > > +};
> > > > +
> > > > +struct acpi_iort_rmr_desc {
> > > > +	u64 base_address;
> > > > +	u64 length;
> > > > +	u32 reserved;
> > > > +};
> > > > +
> > > >
> > > >
> > >
> /**********************************************************
> > > *****
> > > > ****************
> > > >   *
> > > >   * IVRS - I/O Virtualization Reporting Structure
> > > > --
> > > > 2.17.1
> > > >
> > > > _______________________________________________
> > > > iommu mailing list
> > > > iommu@lists.linux-foundation.org
> > > > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> > > _______________________________________________
> > > Devel mailing list -- devel@acpica.org
> > > To unsubscribe send an email to devel-leave@acpica.org
> > > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Lorenzo Pieralisi]

On Tue, Mar 23, 2021 at 06:51:44PM +0000, Kaneda, Erik wrote:
> 
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> > Sent: Tuesday, March 23, 2021 8:54 AM
> > To: Kaneda, Erik <erik.kaneda@intel.com>
> > Cc: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> > linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-foundation.org; devel@acpica.org; Moore, Robert
> > <robert.moore@intel.com>; Linuxarm <linuxarm@huawei.com>;
> > steven.price@arm.com; Sami.Mujawar@arm.com; robin.murphy@arm.com;
> > wanghuiqiang <wanghuiqiang@huawei.com>
> > Subject: Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> > 
> > On Mon, Mar 22, 2021 at 09:57:58PM +0000, Kaneda, Erik wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Shameerali Kolothum Thodi
> > > > <shameerali.kolothum.thodi@huawei.com>
> > > > Sent: Monday, March 22, 2021 3:36 AM
> > > > To: Kaneda, Erik <erik.kaneda@intel.com>; linux-arm-
> > > > kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-
> > > > foundation.org; devel@acpica.org; Lorenzo Pieralisi
> > > > <lorenzo.pieralisi@arm.com>; Moore, Robert
> > <robert.moore@intel.com>
> > > > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > > > Sami.Mujawar@arm.com; robin.murphy@arm.com; wanghuiqiang
> > > > <wanghuiqiang@huawei.com>
> > > > Subject: [Devel] Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for
> > revision E
> > > >
> > > > [+]
> > > >
> > > > Hi Erik,
> > > >
> > > > As this is now just merged ino acpica-master and based on the discussion
> > we
> > > > had here,
> > > >
> > > > https://github.com/acpica/acpica/pull/638
> > > >
> > > > I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions
> > call
> > > > and
> > > > can confirm that the IORT Revision E is not the final specification and has
> > > > some issues
> > > > which is now corrected in the latest E.b revision[1]. Also there are no
> > current
> > > > users
> > > > for the Rev E and it may not be a good idea to push this version into the
> > Linux
> > > > kernel
> > > > or elsewhere.
> > > >
> > > > So could you please revert the merge and I am planning to work on the
> > E.b
> > > > soon.
> > > Hi,
> > >
> > > > Please let me know if I need to explicitly send a revert pull request or not.
> > >
> > > Please send a revert pull request and I'll remember to not submit Linux-ize
> > the IORT patches.
> > >
> > > From all of the activity that I've seen with the IORT specification,
> > > it looks like this table is nontrivial to design and maintain. The
> > > difficulty I have with the table is that I do not understand which
> > > table revisions are in active use.
> > 
> Hi Lorenzo,
> 
> > Possibly all of them in firmware in the field - I am not sure what you
> > are asking though; if you can elaborate I'd be grateful.
> 
> Yes, I'd be happy to explain.
> 
> The ACPICA project contains code that provides definitions for ACPI tables. After each release of this project, the codebase gets translated to a Linux style syntax and relevant patches are submitted to Linux so that it can use these table definitions in their driver codebase. From ACPICA's perspective, the code providing these definitions are used to implement a compiler/disassembler called iASL. This tool disassembles table definitions so that the user doesn't have to open a hex editor to decipher ACPI tables. Our goal with iASL is to be able to disassemble as many ACPI tables as possible to give users the ability to compile and debug ACPI tables.
> 
> Out of the 60+ tables that we support, IORT has been tricky to maintain. There are revisions A-E and we have received pull requests from engineers from ARM or companies that work with ARM. We are grateful of their contributions but some of these pull requests have broken support for earlier versions of IORT. In addition, we sometimes receive communication from people like Shameer saying that revision E does not currently have users. This leaves Bob and I very confused about which revisions we should be focused on supporting in iASL.
> 
> We need your help in understanding which versions of IORT should be supported by iASL as well as Linux.
> 
> I hope this helps.. Let me know if you need me to clarify anything that I've written.

Yes, it does, it is my question that wasn't clear, I maintain the
Linux ACPI ARM64 code, I am familiar with ACPICA and all of the above.

What I wanted to say is that overall, all versions should be supported
and I *think* ACPICA is designed so that the static table headers are
meant to support the *latest* table version (whose firmware bindings are
supposed to be backward compatible with all existing versions "in use").

However, revision E and E.a required a spec update, hence Shameer revert
request (which I support).

I think what you are asking is someone from Arm to act as a gatekeeper
to help you ACK/NAK ACPICA IORT changes because you have no visibility
into IORT specs evolution and actual deployment.

Is my understanding correct ?

Thanks,
Lorenzo

> Thanks,
> Erik
> > 
> > > So my question is this: which IORT revisions are being actively used?
> > 
> > See above.
> > 
> > Thanks,
> > Lorenzo
> > 
> > >
> > > Thanks,
> > > Erik
> > > >
> > > > Thanks,
> > > > Shameer
> > > >
> > > > 1. https://developer.arm.com/documentation/den0049/latest/
> > > >
> > > > > -----Original Message-----
> > > > > From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On
> > > > Behalf Of
> > > > > Shameer Kolothum
> > > > > Sent: 19 November 2020 12:12
> > > > > To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > > > > iommu@lists.linux-foundation.org; devel@acpica.org
> > > > > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com;
> > > > Guohanjun
> > > > > (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> > > > > robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > > > > Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> > > > >
> > > > > IORT revision E contains a few additions like,
> > > > >     -Added an identifier field in the node descriptors to aid table
> > > > >      cross-referencing.
> > > > >     -Introduced the Reserved Memory Range(RMR) node. This is used
> > > > >      to describe memory ranges that are used by endpoints and requires
> > > > >      a unity mapping in SMMU.
> > > > >     -Introduced a flag in the RC node to express support for PRI.
> > > > >
> > > > > Signed-off-by: Shameer Kolothum
> > > > <shameerali.kolothum.thodi@huawei.com>
> > > > > ---
> > > > >  include/acpi/actbl2.h | 25 +++++++++++++++++++------
> > > > >  1 file changed, 19 insertions(+), 6 deletions(-)
> > > > >
> > > > > diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> > > > > ec66779cb193..274fce7b5c01 100644
> > > > > --- a/include/acpi/actbl2.h
> > > > > +++ b/include/acpi/actbl2.h
> > > > > @@ -68,7 +68,7 @@
> > > > >   * IORT - IO Remapping Table
> > > > >   *
> > > > >   * Conforms to "IO Remapping Table System Software on ARM
> > Platforms",
> > > > > - * Document number: ARM DEN 0049D, March 2018
> > > > > + * Document number: ARM DEN 0049E, June 2020
> > > > >   *
> > > > >
> > > > >
> > > >
> > **********************************************************
> > > > ******
> > > > > **************/
> > > > >
> > > > > @@ -86,7 +86,8 @@ struct acpi_iort_node {
> > > > >  	u8 type;
> > > > >  	u16 length;
> > > > >  	u8 revision;
> > > > > -	u32 reserved;
> > > > > +	u16 reserved;
> > > > > +	u16 identifier;
> > > > >  	u32 mapping_count;
> > > > >  	u32 mapping_offset;
> > > > >  	char node_data[1];
> > > > > @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
> > > > >  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
> > > > >  	ACPI_IORT_NODE_SMMU = 0x03,
> > > > >  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> > > > > -	ACPI_IORT_NODE_PMCG = 0x05
> > > > > +	ACPI_IORT_NODE_PMCG = 0x05,
> > > > > +	ACPI_IORT_NODE_RMR = 0x06,
> > > > >  };
> > > > >
> > > > >  struct acpi_iort_id_mapping {
> > > > > @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
> > > > >  	u8 reserved[3];		/* Reserved, must be zero */
> > > > >  };
> > > > >
> > > > > -/* Values for ats_attribute field above */
> > > > > +/* Masks for ats_attribute field above */
> > > > >
> > > > > -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root
> > > > > complex supports ATS */
> > > > > -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root
> > > > > complex doesn't support ATS */
> > > > > +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex
> > > > > supports ATS */
> > > > > +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root
> > complex
> > > > > supports PRI */
> > > > >
> > > > >  struct acpi_iort_smmu {
> > > > >  	u64 base_address;	/* SMMU base address */
> > > > > @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
> > > > >  	u64 page1_base_address;
> > > > >  };
> > > > >
> > > > > +struct acpi_iort_rmr {
> > > > > +	u32 rmr_count;
> > > > > +	u32 rmr_offset;
> > > > > +};
> > > > > +
> > > > > +struct acpi_iort_rmr_desc {
> > > > > +	u64 base_address;
> > > > > +	u64 length;
> > > > > +	u32 reserved;
> > > > > +};
> > > > > +
> > > > >
> > > > >
> > > >
> > /**********************************************************
> > > > *****
> > > > > ****************
> > > > >   *
> > > > >   * IVRS - I/O Virtualization Reporting Structure
> > > > > --
> > > > > 2.17.1
> > > > >
> > > > > _______________________________________________
> > > > > iommu mailing list
> > > > > iommu@lists.linux-foundation.org
> > > > > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> > > > _______________________________________________
> > > > Devel mailing list -- devel@acpica.org
> > > > To unsubscribe send an email to devel-leave@acpica.org
> > > > %(web_page_url)slistinfo%(cgiext)s/%(_internal_name)s

Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Jon Nettleton]

On Mon, Mar 22, 2021 at 11:37 AM Shameerali Kolothum Thodi
<shameerali.kolothum.thodi@huawei.com> wrote:
>
> [+]
>
> Hi Erik,
>
> As this is now just merged ino acpica-master and based on the discussion we had here,
>
> https://github.com/acpica/acpica/pull/638
>
> I had a discussion with ARM folks(Lorenzo) in the linaro-open-discussions call and
> can confirm that the IORT Revision E is not the final specification and has some issues
> which is now corrected in the latest E.b revision[1]. Also there are no current users
> for the Rev E and it may not be a good idea to push this version into the Linux kernel
> or elsewhere.

Well it was a released revision, although it was found to have issues.
Currently
HoneyComb Systems Ready certified firmware does include support for this table,
although incomplete.  Without agreement on mainline support I am fine to update
to the latest spec bump.

>
> So could you please revert the merge and I am planning to work on the E.b soon.
> Please let me know if I need to explicitly send a revert pull request or not.

Can you please CC. me on your next release.  I was planning on spending time
on this regardless.  I already have a patchset for the fsl-mc-bus driver that
needs to change in order to function properly with RMR support.

Thanks.

>
> Thanks,
> Shameer
>
> 1. https://developer.arm.com/documentation/den0049/latest/
>
> > -----Original Message-----
> > From: iommu [mailto:iommu-bounces@lists.linux-foundation.org] On Behalf Of
> > Shameer Kolothum
> > Sent: 19 November 2020 12:12
> > To: linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-foundation.org; devel@acpica.org
> > Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun
> > (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> > robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> > Subject: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E
> >
> > IORT revision E contains a few additions like,
> >     -Added an identifier field in the node descriptors to aid table
> >      cross-referencing.
> >     -Introduced the Reserved Memory Range(RMR) node. This is used
> >      to describe memory ranges that are used by endpoints and requires
> >      a unity mapping in SMMU.
> >     -Introduced a flag in the RC node to express support for PRI.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  include/acpi/actbl2.h | 25 +++++++++++++++++++------
> >  1 file changed, 19 insertions(+), 6 deletions(-)
> >
> > diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h index
> > ec66779cb193..274fce7b5c01 100644
> > --- a/include/acpi/actbl2.h
> > +++ b/include/acpi/actbl2.h
> > @@ -68,7 +68,7 @@
> >   * IORT - IO Remapping Table
> >   *
> >   * Conforms to "IO Remapping Table System Software on ARM Platforms",
> > - * Document number: ARM DEN 0049D, March 2018
> > + * Document number: ARM DEN 0049E, June 2020
> >   *
> >
> > ****************************************************************
> > **************/
> >
> > @@ -86,7 +86,8 @@ struct acpi_iort_node {
> >       u8 type;
> >       u16 length;
> >       u8 revision;
> > -     u32 reserved;
> > +     u16 reserved;
> > +     u16 identifier;
> >       u32 mapping_count;
> >       u32 mapping_offset;
> >       char node_data[1];
> > @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
> >       ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
> >       ACPI_IORT_NODE_SMMU = 0x03,
> >       ACPI_IORT_NODE_SMMU_V3 = 0x04,
> > -     ACPI_IORT_NODE_PMCG = 0x05
> > +     ACPI_IORT_NODE_PMCG = 0x05,
> > +     ACPI_IORT_NODE_RMR = 0x06,
> >  };
> >
> >  struct acpi_iort_id_mapping {
> > @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
> >       u8 reserved[3];         /* Reserved, must be zero */
> >  };
> >
> > -/* Values for ats_attribute field above */
> > +/* Masks for ats_attribute field above */
> >
> > -#define ACPI_IORT_ATS_SUPPORTED         0x00000001   /* The root
> > complex supports ATS */
> > -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000   /* The root
> > complex doesn't support ATS */
> > +#define ACPI_IORT_ATS_SUPPORTED         (1)  /* The root complex
> > supports ATS */
> > +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)       /* The root complex
> > supports PRI */
> >
> >  struct acpi_iort_smmu {
> >       u64 base_address;       /* SMMU base address */
> > @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
> >       u64 page1_base_address;
> >  };
> >
> > +struct acpi_iort_rmr {
> > +     u32 rmr_count;
> > +     u32 rmr_offset;
> > +};
> > +
> > +struct acpi_iort_rmr_desc {
> > +     u64 base_address;
> > +     u64 length;
> > +     u32 reserved;
> > +};
> > +
> >
> > /***************************************************************
> > ****************
> >   *
> >   * IVRS - I/O Virtualization Reporting Structure
> > --
> > 2.17.1
> >
> > _______________________________________________
> > iommu mailing list
> > iommu@lists.linux-foundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/iommu
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

RE: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Jon Nettleton [mailto:jon@solid-run.com]
> Sent: 25 March 2021 08:40
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
> Cc: erik.kaneda@intel.com; linux-arm-kernel@lists.infradead.org;
> linux-acpi@vger.kernel.org; iommu@lists.linux-foundation.org;
> devel@acpica.org; Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>;
> robert.moore@intel.com; Linuxarm <linuxarm@huawei.com>;
> steven.price@arm.com; Guohanjun (Hanjun Guo) <guohanjun@huawei.com>;
> Sami.Mujawar@arm.com; robin.murphy@arm.com; wanghuiqiang
> <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[...]

> Well it was a released revision, although it was found to have issues.
> Currently
> HoneyComb Systems Ready certified firmware does include support for this
> table,
> although incomplete.  Without agreement on mainline support I am fine to
> update
> to the latest spec bump.

Ok. Sorry didn’t know that you already had a firmware with that revision.
Thanks for agreeing to update to latest.

> >
> > So could you please revert the merge and I am planning to work on the E.b
> soon.
> > Please let me know if I need to explicitly send a revert pull request or not.
> 
> Can you please CC. me on your next release.  I was planning on spending time
> on this regardless.  I already have a patchset for the fsl-mc-bus driver that
> needs to change in order to function properly with RMR support.

Sure. Will CC you.

Hi All,

I have send pull requests to acpica git for reverting rev E and to update it with E.b,

https://github.com/acpica/acpica/pull/682

Please take a look and let me know.

Thanks,
Shameer

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Auger Eric]

Hi Shameer,

On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> RFC v1 --> v2:
>  - Added a generic interface for IOMMU drivers to retrieve all the 
>    RMR info associated with a given IOMMU.
>  - SMMUv3 driver gets the RMR list during probe() and installs
>    bypass STEs for all the SIDs in the RMR list. This is to keep
>    the ongoing traffic alive(if any) during SMMUv3 reset. This is
>    based on the suggestions received for v1 to take care of the
>    EFI framebuffer use case. Only sanity tested for now.
>  - During the probe/attach device, SMMUv3 driver reserves any
>    RMR region associated with the device such that there is a unity
>    mapping for them in SMMU.
> ---    
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
> ranges that are used by endpoints and require a unity mapping
> in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these controllers
> make use of host memory for various caching related purposes and when
> SMMU is enabled the iMR firmware fails to access these memory regions
> as there is no mapping for them. IORT RMR provides a way for UEFI to
> describe and report these memory regions so that the kernel can make
> a unity mapping for these in SMMU.
> 
> RFC because, Patch #1 is to update the actbl2.h and should be done
> through acpica update. I have send out a pull request[1] for that.

What is the state of this series? I wondered if I should consider using
it for nested SMMU to avoid handling nested binding for MSI, as
suggested by Jean. Are there any blocker?

Thanks

Eric
> 
> Tests:
> 
> With a UEFI, that reports the RMR for the dev,
> ....
> [16F0h 5872   1]                         Type : 06
> [16F1h 5873   2]                       Length : 007C
> [16F3h 5875   1]                     Revision : 00
> [1038h 0056   2]                     Reserved : 00000000
> [1038h 0056   2]                   Identifier : 00000000
> [16F8h 5880   4]                Mapping Count : 00000001
> [16FCh 5884   4]               Mapping Offset : 00000040
> 
> [1700h 5888   4]    Number of RMR Descriptors : 00000002
> [1704h 5892   4]        RMR Descriptor Offset : 00000018
> 
> [1708h 5896   8]          Base Address of RMR : 0000E6400000
> [1710h 5904   8]                Length of RMR : 000000100000
> [1718h 5912   4]                     Reserved : 00000000
> 
> [171Ch 5916   8]          Base Address of RMR : 0000000027B00000
> [1724h 5924   8]                Length of RMR : 0000000000C00000
> [172Ch 5932   4]                     Reserved : 00000000
> 
> [1730h 5936   4]                   Input base : 00000000
> [1734h 5940   4]                     ID Count : 00000001
> [1738h 5944   4]                  Output Base : 00000003
> [173Ch 5948   4]             Output Reference : 00000064
> [1740h 5952   4]        Flags (decoded below) : 00000001
>                                Single Mapping : 1
> ...
> 
> Without the series the RAID controller initialization fails as
> below,
> 
> ...
> [   12.631117] megaraid_sas 0000:03:00.0: FW supports sync cache        : Yes   
> [   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
> [   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED for SCSI host 0                                                                         
> [   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to ready state 
> [  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault code:0x10000 subcode:0x0 func:megasas_transition_to_ready                                    
> [  106.695186] megaraid_sas 0000:03:00.0: System Register set:                  
> [  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller to ready for scsi0.                                                                   
> [  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw 6407      
> estuary:/$
> 
> With the series, now the kernel has direct mapping for the dev as
> below,
> 
> estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions                      
> 0x0000000008000000 0x00000000080fffff msi                                       
> 0x0000000027b00000 0x00000000286fffff direct                                    
> 0x00000000e6400000 0x00000000e64fffff direct                                    
> estuary:/$
> 
> ....
> [   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
> [   12.739089] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs: 0      max_lds: 32                                                                      
> [   12.746628] megaraid_sas 0000:03:00.0: controller type       : iMR(0MB)      
> [   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  : Enabled                                                                               
> [   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes           
> [   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes           
> [   12.771931] megaraid_sas 0000:03:00.0: FW provided TM TaskAbort/Reset timeou: 6 secs/60 secs                                                                 
> [   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map support     : Yes   
> [   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining support    : No    
> [   12.819179] megaraid_sas 0000:03:00.0: NVME page size        : (4096)        
> [   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is called outbound_intr_mask:0x40000000                                                    
> [   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done                     
> [   12.873932] megaraid_sas 0000:03:00.0: pci id                : (0x1000)/(0x0017)/(0x19e5)/(0xd213)                                                           
> [   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no            
> [   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no            
> [   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     : enabled       
> 
> RAID controller init is now success and can detect the drives
> attached as well.
> 
> Thanks,
> Shameer
> 
> [0]. https://developer.arm.com/documentation/den0049/latest/
> [1]. https://github.com/acpica/acpica/pull/638
> 
> Shameer Kolothum (8):
>   ACPICA: IORT: Update for revision E
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add RMR memory regions reservation helper
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev
> 
>  drivers/acpi/arm64/iort.c                   | 182 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 112 ++++++++++--
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   2 +
>  drivers/iommu/dma-iommu.c                   |  39 +++++
>  include/acpi/actbl2.h                       |  25 ++-
>  include/linux/acpi_iort.h                   |   6 +
>  include/linux/dma-iommu.h                   |   7 +
>  include/linux/iommu.h                       |  16 ++
>  8 files changed, 367 insertions(+), 22 deletions(-)
> 

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

Hi Eric,

> -----Original Message-----
> From: Auger Eric [mailto:eric.auger@redhat.com]
> Sent: 09 April 2021 10:51
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> Hi Shameer,
> 
> On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> > RFC v1 --> v2:
> >  - Added a generic interface for IOMMU drivers to retrieve all the
> >    RMR info associated with a given IOMMU.
> >  - SMMUv3 driver gets the RMR list during probe() and installs
> >    bypass STEs for all the SIDs in the RMR list. This is to keep
> >    the ongoing traffic alive(if any) during SMMUv3 reset. This is
> >    based on the suggestions received for v1 to take care of the
> >    EFI framebuffer use case. Only sanity tested for now.
> >  - During the probe/attach device, SMMUv3 driver reserves any
> >    RMR region associated with the device such that there is a unity
> >    mapping for them in SMMU.
> > ---
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
> > ranges that are used by endpoints and require a unity mapping
> > in SMMU.
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these controllers
> > make use of host memory for various caching related purposes and when
> > SMMU is enabled the iMR firmware fails to access these memory regions
> > as there is no mapping for them. IORT RMR provides a way for UEFI to
> > describe and report these memory regions so that the kernel can make
> > a unity mapping for these in SMMU.
> >
> > RFC because, Patch #1 is to update the actbl2.h and should be done
> > through acpica update. I have send out a pull request[1] for that.
> 
> What is the state of this series? I wondered if I should consider using
> it for nested SMMU to avoid handling nested binding for MSI, as
> suggested by Jean. Are there any blocker?

There were few issues with the revision E spec and those are now sorted 
with an updated E.b.

The ACPICA pull request for E.b is now merged[1] and the Linux header
update patch[2] is also out there(I think it is now queued for 5.13).

I will soon respin this series.

Thanks,
Shameer

1. https://github.com/acpica/acpica/pull/682
2. https://lore.kernel.org/linux-acpi/20210406213028.718796-22-erik.kaneda@intel.com/

> 
> Thanks
> 
> Eric
> >
> > Tests:
> >
> > With a UEFI, that reports the RMR for the dev,
> > ....
> > [16F0h 5872   1]                         Type : 06
> > [16F1h 5873   2]                       Length : 007C
> > [16F3h 5875   1]                     Revision : 00
> > [1038h 0056   2]                     Reserved : 00000000
> > [1038h 0056   2]                   Identifier : 00000000
> > [16F8h 5880   4]                Mapping Count : 00000001
> > [16FCh 5884   4]               Mapping Offset : 00000040
> >
> > [1700h 5888   4]    Number of RMR Descriptors : 00000002
> > [1704h 5892   4]        RMR Descriptor Offset : 00000018
> >
> > [1708h 5896   8]          Base Address of RMR : 0000E6400000
> > [1710h 5904   8]                Length of RMR : 000000100000
> > [1718h 5912   4]                     Reserved : 00000000
> >
> > [171Ch 5916   8]          Base Address of RMR : 0000000027B00000
> > [1724h 5924   8]                Length of RMR : 0000000000C00000
> > [172Ch 5932   4]                     Reserved : 00000000
> >
> > [1730h 5936   4]                   Input base : 00000000
> > [1734h 5940   4]                     ID Count : 00000001
> > [1738h 5944   4]                  Output Base : 00000003
> > [173Ch 5948   4]             Output Reference : 00000064
> > [1740h 5952   4]        Flags (decoded below) : 00000001
> >                                Single Mapping : 1
> > ...
> >
> > Without the series the RAID controller initialization fails as
> > below,
> >
> > ...
> > [   12.631117] megaraid_sas 0000:03:00.0: FW supports sync
> cache        : Yes
> > [   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is
> called outbound_intr_mask:0x40000009
> > [   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED
> for SCSI host 0
> > [   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to
> ready state
> > [  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault
> code:0x10000 subcode:0x0 func:megasas_transition_to_ready
> > [  106.695186] megaraid_sas 0000:03:00.0: System Register set:
> > [  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller to
> ready for scsi0.
> > [  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw
> 6407
> > estuary:/$
> >
> > With the series, now the kernel has direct mapping for the dev as
> > below,
> >
> > estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions
> > 0x0000000008000000 0x00000000080fffff msi
> > 0x0000000027b00000 0x00000000286fffff direct
> > 0x00000000e6400000 0x00000000e64fffff direct
> > estuary:/$
> >
> > ....
> > [   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is
> called outbound_intr_mask:0x40000009
> > [   12.739089] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs:
> 0      max_lds: 32
> > [   12.746628] megaraid_sas 0000:03:00.0: controller type       :
> iMR(0MB)
> > [   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  :
> Enabled
> > [   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes
> > [   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes
> > [   12.771931] megaraid_sas 0000:03:00.0: FW provided TM
> TaskAbort/Reset timeou: 6 secs/60 secs
> > [   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map
> support     : Yes
> > [   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining
> support    : No
> > [   12.819179] megaraid_sas 0000:03:00.0: NVME page size        :
> (4096)
> > [   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is
> called outbound_intr_mask:0x40000000
> > [   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done
> > [   12.873932] megaraid_sas 0000:03:00.0: pci id                :
> (0x1000)/(0x0017)/(0x19e5)/(0xd213)
> > [   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no
> > [   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no
> > [   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     :
> enabled
> >
> > RAID controller init is now success and can detect the drives
> > attached as well.
> >
> > Thanks,
> > Shameer
> >
> > [0]. https://developer.arm.com/documentation/den0049/latest/
> > [1]. https://github.com/acpica/acpica/pull/638
> >
> > Shameer Kolothum (8):
> >   ACPICA: IORT: Update for revision E
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add RMR memory regions reservation helper
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 182
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 112 ++++++++++--
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   2 +
> >  drivers/iommu/dma-iommu.c                   |  39 +++++
> >  include/acpi/actbl2.h                       |  25 ++-
> >  include/linux/acpi_iort.h                   |   6 +
> >  include/linux/dma-iommu.h                   |   7 +
> >  include/linux/iommu.h                       |  16 ++
> >  8 files changed, 367 insertions(+), 22 deletions(-)
> >

Re: [RFC PATCH v2 1/8] ACPICA: IORT: Update for revision E

[Auger Eric]

Hi Shameer,

On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> IORT revision E contains a few additions like,
>     -Added an identifier field in the node descriptors to aid table
>      cross-referencing.
>     -Introduced the Reserved Memory Range(RMR) node. This is used
>      to describe memory ranges that are used by endpoints and requires
>      a unity mapping in SMMU.
>     -Introduced a flag in the RC node to express support for PRI.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  include/acpi/actbl2.h | 25 +++++++++++++++++++------
>  1 file changed, 19 insertions(+), 6 deletions(-)
> 
> diff --git a/include/acpi/actbl2.h b/include/acpi/actbl2.h
> index ec66779cb193..274fce7b5c01 100644
> --- a/include/acpi/actbl2.h
> +++ b/include/acpi/actbl2.h
> @@ -68,7 +68,7 @@
>   * IORT - IO Remapping Table
>   *
>   * Conforms to "IO Remapping Table System Software on ARM Platforms",
> - * Document number: ARM DEN 0049D, March 2018
> + * Document number: ARM DEN 0049E, June 2020
>   *
>   ******************************************************************************/
>  
> @@ -86,7 +86,8 @@ struct acpi_iort_node {
>  	u8 type;
>  	u16 length;
>  	u8 revision;
> -	u32 reserved;
> +	u16 reserved;
> +	u16 identifier;
>  	u32 mapping_count;
>  	u32 mapping_offset;
>  	char node_data[1];
> @@ -100,7 +101,8 @@ enum acpi_iort_node_type {
>  	ACPI_IORT_NODE_PCI_ROOT_COMPLEX = 0x02,
>  	ACPI_IORT_NODE_SMMU = 0x03,
>  	ACPI_IORT_NODE_SMMU_V3 = 0x04,
> -	ACPI_IORT_NODE_PMCG = 0x05
> +	ACPI_IORT_NODE_PMCG = 0x05,
> +	ACPI_IORT_NODE_RMR = 0x06,
>  };
>  
>  struct acpi_iort_id_mapping {
> @@ -167,10 +169,10 @@ struct acpi_iort_root_complex {
>  	u8 reserved[3];		/* Reserved, must be zero */
>  };
>  
> -/* Values for ats_attribute field above */
> +/* Masks for ats_attribute field above */
>  
> -#define ACPI_IORT_ATS_SUPPORTED         0x00000001	/* The root complex supports ATS */
> -#define ACPI_IORT_ATS_UNSUPPORTED       0x00000000	/* The root complex doesn't support ATS */
> +#define ACPI_IORT_ATS_SUPPORTED         (1)	/* The root complex supports ATS */
> +#define ACPI_IORT_PRI_SUPPORTED         (1<<1)	/* The root complex supports PRI */
>  
>  struct acpi_iort_smmu {
>  	u64 base_address;	/* SMMU base address */
> @@ -241,6 +243,17 @@ struct acpi_iort_pmcg {
>  	u64 page1_base_address;
>  };
>  
> +struct acpi_iort_rmr {
so indeed in E.b there is a new field here.
u32 flags
> +	u32 rmr_count;
> +	u32 rmr_offset;
> +};
> +
> +struct acpi_iort_rmr_desc {
> +	u64 base_address;
> +	u64 length;
> +	u32 reserved;
> +};
> +
>  /*******************************************************************************
>   *
>   * IVRS - I/O Virtualization Reporting Structure
> 
Thanks

Eric

Re: [RFC PATCH v2 2/8] ACPI/IORT: Add support for RMR node parsing

[Auger Eric]

Hi Shameer,
On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> Add support for parsing RMR node information from ACPI.
> Find associated stream ids and smmu node info from the
> RMR node and populate a linked list with RMR memory
> descriptors.
> 
> Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c | 122 +++++++++++++++++++++++++++++++++++++-
>  1 file changed, 121 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index 9929ff50c0c0..a9705aa35028 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -40,6 +40,25 @@ struct iort_fwnode {
>  static LIST_HEAD(iort_fwnode_list);
>  static DEFINE_SPINLOCK(iort_fwnode_lock);
>  
> +struct iort_rmr_id {
> +	u32  sid;
> +	struct acpi_iort_node *smmu;
> +};
> +
> +/*
> + * One entry for IORT RMR.
> + */
> +struct iort_rmr_entry {
> +	struct list_head list;
> +
> +	unsigned int rmr_ids_num;
> +	struct iort_rmr_id *rmr_ids;
> +
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +};
> +
> +static LIST_HEAD(iort_rmr_list);         /* list of RMR regions from ACPI */
> +
>  /**
>   * iort_set_fwnode() - Create iort_fwnode and use it to register
>   *		       iommu data in the iort_fwnode_list
> @@ -393,7 +412,8 @@ static struct acpi_iort_node *iort_node_get_id(struct acpi_iort_node *node,
>  		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
>  		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
>  		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> -		    node->type == ACPI_IORT_NODE_PMCG) {
> +		    node->type == ACPI_IORT_NODE_PMCG ||
> +		    node->type == ACPI_IORT_NODE_RMR) {
>  			*id_out = map->output_base;
>  			return parent;
>  		}
> @@ -1647,6 +1667,103 @@ static void __init iort_enable_acs(struct acpi_iort_node *iort_node)
>  #else
>  static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
>  #endif
> +static int iort_rmr_desc_valid(struct acpi_iort_rmr_desc *desc)
> +{
> +	struct iort_rmr_entry *e;
> +	u64 end, start = desc->base_address, length = desc->length;
> +
> +	if (!IS_ALIGNED(start, SZ_64K) || !IS_ALIGNED(length, SZ_64K))
> +		return -EINVAL;
> +
> +	end = start + length - 1;
> +
> +	/* Check for address overlap */
I don't get this check. What is the problem if you attach the same range
to different stream ids. Shouldn't you check there is no overlap for the
same sid?


> +	list_for_each_entry(e, &iort_rmr_list, list) {
> +		u64 e_start = e->rmr_desc->base_address;
> +		u64 e_end = e_start + e->rmr_desc->length - 1;
> +
> +		if (start <= e_end && end >= e_start)
> +			return -EINVAL;
> +	}
> +
> +	return 0;
> +}
> +
> +static int __init iort_parse_rmr(struct acpi_iort_node *iort_node)
> +{
> +	struct iort_rmr_id *rmr_ids, *ids;
> +	struct iort_rmr_entry *e;
> +	struct acpi_iort_rmr *rmr;
> +	struct acpi_iort_rmr_desc *rmr_desc;
> +	u32 map_count = iort_node->mapping_count;
> +	int i, ret = 0, desc_count = 0;
> +
> +	if (iort_node->type != ACPI_IORT_NODE_RMR)
> +		return 0;
> +
> +	if (!iort_node->mapping_offset || !map_count) {
> +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> +		       iort_node);
> +		return -EINVAL;
> +	}
> +
> +	rmr_ids = kmalloc(sizeof(*rmr_ids) * map_count, GFP_KERNEL);
> +	if (!rmr_ids)
> +		return -ENOMEM;
> +
> +	/* Retrieve associated smmu and stream id */
> +	ids = rmr_ids;
nit: do you need both rmr_ids and ids?
> +	for (i = 0; i < map_count; i++, ids++) {
> +		ids->smmu = iort_node_get_id(iort_node, &ids->sid, i);
> +		if (!ids->smmu) {
> +			pr_err(FW_BUG "Invalid SMMU reference, skipping RMR node %p\n",
> +			       iort_node);
> +			ret = -EINVAL;
> +			goto out;
> +		}
> +	}
> +
> +	/* Retrieve RMR data */
> +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR node %p\n",
> +		       iort_node);
> +		ret = -EINVAL;
> +		goto out;
> +	}
> +
> +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> +				rmr->rmr_offset);
> +
> +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> +		ret = iort_rmr_desc_valid(rmr_desc);
> +		if (ret) {
> +			pr_err(FW_BUG "Invalid RMR descriptor[%d] for node %p, skipping...\n",
> +			       i, iort_node);
> +			goto out;
so I understand you skip the whole node and not just that rmr desc,
otherwise you would continue. so in that case don't you need to free
both rmr_ids and already allocated 'e'?
> +		}
> +
> +		e = kmalloc(sizeof(*e), GFP_KERNEL);
> +		if (!e) {
> +			ret = -ENOMEM;
> +			goto out;
> +		}
> +
> +		e->rmr_ids_num = map_count;
> +		e->rmr_ids = rmr_ids;
> +		e->rmr_desc = rmr_desc;
> +
> +		list_add_tail(&e->list, &iort_rmr_list);
> +		desc_count++;
> +	}
> +
> +	return 0;
> +
> +out:
> +	if (!desc_count)
don't you want to test ret instead? see comment above. + free allocated ''e'
> +		kfree(rmr_ids);
> +	return ret;
> +}
>  
>  static void __init iort_init_platform_devices(void)
>  {
> @@ -1676,6 +1793,9 @@ static void __init iort_init_platform_devices(void)
>  
>  		iort_enable_acs(iort_node);
>  
> +		if (iort_table->revision == 1)
> +			iort_parse_rmr(iort_node);
> +
>  		ops = iort_get_dev_cfg(iort_node);
>  		if (ops) {
>  			fwnode = acpi_alloc_fwnode_static();
> 
Thanks

Eric

Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Auger Eric]

Hi Shameer,

+ Jean-Philippe


On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> RFC v1 --> v2:
>  - Added a generic interface for IOMMU drivers to retrieve all the 
>    RMR info associated with a given IOMMU.
>  - SMMUv3 driver gets the RMR list during probe() and installs
>    bypass STEs for all the SIDs in the RMR list. This is to keep
>    the ongoing traffic alive(if any) during SMMUv3 reset. This is
>    based on the suggestions received for v1 to take care of the
>    EFI framebuffer use case. Only sanity tested for now.
>  - During the probe/attach device, SMMUv3 driver reserves any
>    RMR region associated with the device such that there is a unity
>    mapping for them in SMMU.
> ---    
> 
> The series adds support to IORT RMR nodes specified in IORT
> Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
> ranges that are used by endpoints and require a unity mapping
> in SMMU.
> 
> We have faced issues with 3408iMR RAID controller cards which
> fail to boot when SMMU is enabled. This is because these controllers
> make use of host memory for various caching related purposes and when
> SMMU is enabled the iMR firmware fails to access these memory regions
> as there is no mapping for them. IORT RMR provides a way for UEFI to
> describe and report these memory regions so that the kernel can make
> a unity mapping for these in SMMU.
> 
> RFC because, Patch #1 is to update the actbl2.h and should be done
> through acpica update. I have send out a pull request[1] for that.
> 
> Tests:
> 
> With a UEFI, that reports the RMR for the dev,
> ....
> [16F0h 5872   1]                         Type : 06
> [16F1h 5873   2]                       Length : 007C
> [16F3h 5875   1]                     Revision : 00
> [1038h 0056   2]                     Reserved : 00000000
> [1038h 0056   2]                   Identifier : 00000000
> [16F8h 5880   4]                Mapping Count : 00000001
> [16FCh 5884   4]               Mapping Offset : 00000040
> 
> [1700h 5888   4]    Number of RMR Descriptors : 00000002
> [1704h 5892   4]        RMR Descriptor Offset : 00000018
> 
> [1708h 5896   8]          Base Address of RMR : 0000E6400000
> [1710h 5904   8]                Length of RMR : 000000100000
> [1718h 5912   4]                     Reserved : 00000000
> 
> [171Ch 5916   8]          Base Address of RMR : 0000000027B00000
> [1724h 5924   8]                Length of RMR : 0000000000C00000
> [172Ch 5932   4]                     Reserved : 00000000
> 
> [1730h 5936   4]                   Input base : 00000000
> [1734h 5940   4]                     ID Count : 00000001
> [1738h 5944   4]                  Output Base : 00000003
> [173Ch 5948   4]             Output Reference : 00000064
> [1740h 5952   4]        Flags (decoded below) : 00000001
>                                Single Mapping : 1

Following Jean-Philippe's suggestion I have used your series for nested
stage SMMUv3 integration, ie. to simplify the MSI nested stage mapping.

Host allocates hIOVA -> physical doorbell (pDB) as it normally does for
VFIO device passthrough. IOVA Range is 0x8000000 - 0x8100000.

I expose this MIS IOVA range to the guest as an RMR and as a result
guest has a flat mapping for this range. As the physical device is
programmed with hIOVA we have the following mapping:

IOVA            IPA          PA
hIOVA   ->     hIOVA     ->  pDB
        S1               s2

This works.

The only weird thing is that I need to expose 256 RMRs due to the
'Single Mapping' mandatory flag. I need to have 1 RMR per potential SID
on the bus.

I will post a new version of SMMUv3 nested stage soon for people to test
& compare. Obviously this removes a bunch of code on both SMMU/VFIO and
QEMU code so I think this solution looks better overall.

Thanks

Eric
> ...
> 
> Without the series the RAID controller initialization fails as
> below,
> 
> ...
> [   12.631117] megaraid_sas 0000:03:00.0: FW supports sync cache        : Yes   
> [   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
> [   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED for SCSI host 0                                                                         
> [   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to ready state 
> [  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault code:0x10000 subcode:0x0 func:megasas_transition_to_ready                                    
> [  106.695186] megaraid_sas 0000:03:00.0: System Register set:                  
> [  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller to ready for scsi0.                                                                   
> [  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw 6407      
> estuary:/$
> 
> With the series, now the kernel has direct mapping for the dev as
> below,
> 
> estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions                      
> 0x0000000008000000 0x00000000080fffff msi                                       
> 0x0000000027b00000 0x00000000286fffff direct                                    
> 0x00000000e6400000 0x00000000e64fffff direct                                    
> estuary:/$
> 
> ....
> [   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is called outbound_intr_mask:0x40000009                                                   
> [   12.739089] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs: 0      max_lds: 32                                                                      
> [   12.746628] megaraid_sas 0000:03:00.0: controller type       : iMR(0MB)      
> [   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  : Enabled                                                                               
> [   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes           
> [   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes           
> [   12.771931] megaraid_sas 0000:03:00.0: FW provided TM TaskAbort/Reset timeou: 6 secs/60 secs                                                                 
> [   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map support     : Yes   
> [   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining support    : No    
> [   12.819179] megaraid_sas 0000:03:00.0: NVME page size        : (4096)        
> [   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is called outbound_intr_mask:0x40000000                                                    
> [   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done                     
> [   12.873932] megaraid_sas 0000:03:00.0: pci id                : (0x1000)/(0x0017)/(0x19e5)/(0xd213)                                                           
> [   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no            
> [   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no            
> [   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     : enabled       
> 
> RAID controller init is now success and can detect the drives
> attached as well.
> 
> Thanks,
> Shameer
> 
> [0]. https://developer.arm.com/documentation/den0049/latest/
> [1]. https://github.com/acpica/acpica/pull/638
> 
> Shameer Kolothum (8):
>   ACPICA: IORT: Update for revision E
>   ACPI/IORT: Add support for RMR node parsing
>   iommu/dma: Introduce generic helper to retrieve RMR info
>   ACPI/IORT: Add RMR memory regions reservation helper
>   iommu/arm-smmu-v3: Introduce strtab init helper
>   iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
>   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
>   iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev
> 
>  drivers/acpi/arm64/iort.c                   | 182 +++++++++++++++++++-
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 112 ++++++++++--
>  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   2 +
>  drivers/iommu/dma-iommu.c                   |  39 +++++
>  include/acpi/actbl2.h                       |  25 ++-
>  include/linux/acpi_iort.h                   |   6 +
>  include/linux/dma-iommu.h                   |   7 +
>  include/linux/iommu.h                       |  16 ++
>  8 files changed, 367 insertions(+), 22 deletions(-)
> 

RE: [RFC PATCH v2 2/8] ACPI/IORT: Add support for RMR node parsing

[Shameerali Kolothum Thodi]

Hi Eric,

> -----Original Message-----
> From: Auger Eric [mailto:eric.auger@redhat.com]
> Sent: 15 April 2021 10:39
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>
> Subject: Re: [RFC PATCH v2 2/8] ACPI/IORT: Add support for RMR node parsing
> 
> Hi Shameer,
> On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> > Add support for parsing RMR node information from ACPI.
> > Find associated stream ids and smmu node info from the
> > RMR node and populate a linked list with RMR memory
> > descriptors.
> >
> > Signed-off-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  drivers/acpi/arm64/iort.c | 122
> +++++++++++++++++++++++++++++++++++++-
> >  1 file changed, 121 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index 9929ff50c0c0..a9705aa35028 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -40,6 +40,25 @@ struct iort_fwnode {
> >  static LIST_HEAD(iort_fwnode_list);
> >  static DEFINE_SPINLOCK(iort_fwnode_lock);
> >
> > +struct iort_rmr_id {
> > +	u32  sid;
> > +	struct acpi_iort_node *smmu;
> > +};
> > +
> > +/*
> > + * One entry for IORT RMR.
> > + */
> > +struct iort_rmr_entry {
> > +	struct list_head list;
> > +
> > +	unsigned int rmr_ids_num;
> > +	struct iort_rmr_id *rmr_ids;
> > +
> > +	struct acpi_iort_rmr_desc *rmr_desc;
> > +};
> > +
> > +static LIST_HEAD(iort_rmr_list);         /* list of RMR regions from ACPI
> */
> > +
> >  /**
> >   * iort_set_fwnode() - Create iort_fwnode and use it to register
> >   *		       iommu data in the iort_fwnode_list
> > @@ -393,7 +412,8 @@ static struct acpi_iort_node
> *iort_node_get_id(struct acpi_iort_node *node,
> >  		if (node->type == ACPI_IORT_NODE_NAMED_COMPONENT ||
> >  		    node->type == ACPI_IORT_NODE_PCI_ROOT_COMPLEX ||
> >  		    node->type == ACPI_IORT_NODE_SMMU_V3 ||
> > -		    node->type == ACPI_IORT_NODE_PMCG) {
> > +		    node->type == ACPI_IORT_NODE_PMCG ||
> > +		    node->type == ACPI_IORT_NODE_RMR) {
> >  			*id_out = map->output_base;
> >  			return parent;
> >  		}
> > @@ -1647,6 +1667,103 @@ static void __init iort_enable_acs(struct
> acpi_iort_node *iort_node)
> >  #else
> >  static inline void iort_enable_acs(struct acpi_iort_node *iort_node) { }
> >  #endif
> > +static int iort_rmr_desc_valid(struct acpi_iort_rmr_desc *desc)
> > +{
> > +	struct iort_rmr_entry *e;
> > +	u64 end, start = desc->base_address, length = desc->length;
> > +
> > +	if (!IS_ALIGNED(start, SZ_64K) || !IS_ALIGNED(length, SZ_64K))
> > +		return -EINVAL;
> > +
> > +	end = start + length - 1;
> > +
> > +	/* Check for address overlap */
> I don't get this check. What is the problem if you attach the same range
> to different stream ids. Shouldn't you check there is no overlap for the
> same sid?

That’s right. The check should be for memory descriptors within an RMR.
I got confused by the wordings in the IORT spec and that is now clarified
with Lorenzo here,

https://op-lists.linaro.org/pipermail/linaro-open-discussions/2021-April/000150.html

I will change this.
> 
> 
> > +	list_for_each_entry(e, &iort_rmr_list, list) {
> > +		u64 e_start = e->rmr_desc->base_address;
> > +		u64 e_end = e_start + e->rmr_desc->length - 1;
> > +
> > +		if (start <= e_end && end >= e_start)
> > +			return -EINVAL;
> > +	}
> > +
> > +	return 0;
> > +}
> > +
> > +static int __init iort_parse_rmr(struct acpi_iort_node *iort_node)
> > +{
> > +	struct iort_rmr_id *rmr_ids, *ids;
> > +	struct iort_rmr_entry *e;
> > +	struct acpi_iort_rmr *rmr;
> > +	struct acpi_iort_rmr_desc *rmr_desc;
> > +	u32 map_count = iort_node->mapping_count;
> > +	int i, ret = 0, desc_count = 0;
> > +
> > +	if (iort_node->type != ACPI_IORT_NODE_RMR)
> > +		return 0;
> > +
> > +	if (!iort_node->mapping_offset || !map_count) {
> > +		pr_err(FW_BUG "Invalid ID mapping, skipping RMR node %p\n",
> > +		       iort_node);
> > +		return -EINVAL;
> > +	}
> > +
> > +	rmr_ids = kmalloc(sizeof(*rmr_ids) * map_count, GFP_KERNEL);
> > +	if (!rmr_ids)
> > +		return -ENOMEM;
> > +
> > +	/* Retrieve associated smmu and stream id */
> > +	ids = rmr_ids;
> nit: do you need both rmr_ids and ids?

Not really as the spec says it is M:1 mapping. So we only will have one 
single id here(also map_count for the node must be set to 1 as well).

> > +	for (i = 0; i < map_count; i++, ids++) {
> > +		ids->smmu = iort_node_get_id(iort_node, &ids->sid, i);
> > +		if (!ids->smmu) {
> > +			pr_err(FW_BUG "Invalid SMMU reference, skipping RMR
> node %p\n",
> > +			       iort_node);
> > +			ret = -EINVAL;
> > +			goto out;
> > +		}
> > +	}
> > +
> > +	/* Retrieve RMR data */
> > +	rmr = (struct acpi_iort_rmr *)iort_node->node_data;
> > +	if (!rmr->rmr_offset || !rmr->rmr_count) {
> > +		pr_err(FW_BUG "Invalid RMR descriptor array, skipping RMR
> node %p\n",
> > +		       iort_node);
> > +		ret = -EINVAL;
> > +		goto out;
> > +	}
> > +
> > +	rmr_desc = ACPI_ADD_PTR(struct acpi_iort_rmr_desc, iort_node,
> > +				rmr->rmr_offset);
> > +
> > +	for (i = 0; i < rmr->rmr_count; i++, rmr_desc++) {
> > +		ret = iort_rmr_desc_valid(rmr_desc);
> > +		if (ret) {
> > +			pr_err(FW_BUG "Invalid RMR descriptor[%d] for node %p,
> skipping...\n",
> > +			       i, iort_node);
> > +			goto out;
> so I understand you skip the whole node and not just that rmr desc,
> otherwise you would continue. so in that case don't you need to free
> both rmr_ids and already allocated 'e'?

Agree. This needs to be changed.

> > +		}
> > +
> > +		e = kmalloc(sizeof(*e), GFP_KERNEL);
> > +		if (!e) {
> > +			ret = -ENOMEM;
> > +			goto out;
> > +		}
> > +
> > +		e->rmr_ids_num = map_count;
> > +		e->rmr_ids = rmr_ids;
> > +		e->rmr_desc = rmr_desc;
> > +
> > +		list_add_tail(&e->list, &iort_rmr_list);
> > +		desc_count++;
> > +	}
> > +
> > +	return 0;
> > +
> > +out:
> > +	if (!desc_count)
> don't you want to test ret instead? see comment above. + free allocated ''e'

Right. I will change it next revision.

Thanks for taking a look.

Regards,
Shameer

> > +		kfree(rmr_ids);
> > +	return ret;
> > +}
> >
> >  static void __init iort_init_platform_devices(void)
> >  {
> > @@ -1676,6 +1793,9 @@ static void __init iort_init_platform_devices(void)
> >
> >  		iort_enable_acs(iort_node);
> >
> > +		if (iort_table->revision == 1)
> > +			iort_parse_rmr(iort_node);
> > +
> >  		ops = iort_get_dev_cfg(iort_node);
> >  		if (ops) {
> >  			fwnode = acpi_alloc_fwnode_static();
> >
> Thanks
> 
> Eric

RE: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node

[Shameerali Kolothum Thodi]

[+Lorenzo]

> -----Original Message-----
> From: Auger Eric [mailto:eric.auger@redhat.com]
> Sent: 15 April 2021 10:49
> To: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>;
> linux-arm-kernel@lists.infradead.org; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; devel@acpica.org
> Cc: Linuxarm <linuxarm@huawei.com>; steven.price@arm.com; Guohanjun
> (Hanjun Guo) <guohanjun@huawei.com>; Sami.Mujawar@arm.com;
> robin.murphy@arm.com; wanghuiqiang <wanghuiqiang@huawei.com>;
> Jean-Philippe Brucker <jean-philippe@linaro.org>
> Subject: Re: [RFC PATCH v2 0/8] ACPI/IORT: Support for IORT RMR node
> 
> Hi Shameer,
> 
> + Jean-Philippe
> 
> 
> On 11/19/20 1:11 PM, Shameer Kolothum wrote:
> > RFC v1 --> v2:
> >  - Added a generic interface for IOMMU drivers to retrieve all the
> >    RMR info associated with a given IOMMU.
> >  - SMMUv3 driver gets the RMR list during probe() and installs
> >    bypass STEs for all the SIDs in the RMR list. This is to keep
> >    the ongoing traffic alive(if any) during SMMUv3 reset. This is
> >    based on the suggestions received for v1 to take care of the
> >    EFI framebuffer use case. Only sanity tested for now.
> >  - During the probe/attach device, SMMUv3 driver reserves any
> >    RMR region associated with the device such that there is a unity
> >    mapping for them in SMMU.
> > ---
> >
> > The series adds support to IORT RMR nodes specified in IORT
> > Revision E -ARM DEN 0049E[0]. RMR nodes are used to describe memory
> > ranges that are used by endpoints and require a unity mapping
> > in SMMU.
> >
> > We have faced issues with 3408iMR RAID controller cards which
> > fail to boot when SMMU is enabled. This is because these controllers
> > make use of host memory for various caching related purposes and when
> > SMMU is enabled the iMR firmware fails to access these memory regions
> > as there is no mapping for them. IORT RMR provides a way for UEFI to
> > describe and report these memory regions so that the kernel can make
> > a unity mapping for these in SMMU.
> >
> > RFC because, Patch #1 is to update the actbl2.h and should be done
> > through acpica update. I have send out a pull request[1] for that.
> >
> > Tests:
> >
> > With a UEFI, that reports the RMR for the dev,
> > ....
> > [16F0h 5872   1]                         Type : 06
> > [16F1h 5873   2]                       Length : 007C
> > [16F3h 5875   1]                     Revision : 00
> > [1038h 0056   2]                     Reserved : 00000000
> > [1038h 0056   2]                   Identifier : 00000000
> > [16F8h 5880   4]                Mapping Count : 00000001
> > [16FCh 5884   4]               Mapping Offset : 00000040
> >
> > [1700h 5888   4]    Number of RMR Descriptors : 00000002
> > [1704h 5892   4]        RMR Descriptor Offset : 00000018
> >
> > [1708h 5896   8]          Base Address of RMR : 0000E6400000
> > [1710h 5904   8]                Length of RMR : 000000100000
> > [1718h 5912   4]                     Reserved : 00000000
> >
> > [171Ch 5916   8]          Base Address of RMR : 0000000027B00000
> > [1724h 5924   8]                Length of RMR : 0000000000C00000
> > [172Ch 5932   4]                     Reserved : 00000000
> >
> > [1730h 5936   4]                   Input base : 00000000
> > [1734h 5940   4]                     ID Count : 00000001
> > [1738h 5944   4]                  Output Base : 00000003
> > [173Ch 5948   4]             Output Reference : 00000064
> > [1740h 5952   4]        Flags (decoded below) : 00000001
> >                                Single Mapping : 1
> 
> Following Jean-Philippe's suggestion I have used your series for nested
> stage SMMUv3 integration, ie. to simplify the MSI nested stage mapping.
> 
> Host allocates hIOVA -> physical doorbell (pDB) as it normally does for
> VFIO device passthrough. IOVA Range is 0x8000000 - 0x8100000.
> 
> I expose this MIS IOVA range to the guest as an RMR and as a result
> guest has a flat mapping for this range. As the physical device is
> programmed with hIOVA we have the following mapping:
> 
> IOVA            IPA          PA
> hIOVA   ->     hIOVA     ->  pDB
>         S1               s2
> 
> This works.
> 
> The only weird thing is that I need to expose 256 RMRs due to the
> 'Single Mapping' mandatory flag. I need to have 1 RMR per potential SID
> on the bus.

Please see the discussion here,
https://op-lists.linaro.org/pipermail/linaro-open-discussions/2021-April/000150.html

Hi Lorenzo,

May be this is a use case for relaxing that single mapping requirement.

Thanks,
Shameer

> 
> I will post a new version of SMMUv3 nested stage soon for people to test
> & compare. Obviously this removes a bunch of code on both SMMU/VFIO and
> QEMU code so I think this solution looks better overall.
> 
> Thanks
> 
> Eric
> > ...
> >
> > Without the series the RAID controller initialization fails as
> > below,
> >
> > ...
> > [   12.631117] megaraid_sas 0000:03:00.0: FW supports sync
> cache        : Yes
> > [   12.637360] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is
> called outbound_intr_mask:0x40000009
> > [   18.776377] megaraid_sas 0000:03:00.0: Init cmd return status FAILED
> for SCSI host 0
> > [   23.019383] megaraid_sas 0000:03:00.0: Waiting for FW to come to
> ready state
> > [  106.684281] megaraid_sas 0000:03:00.0: FW in FAULT state, Fault
> code:0x10000 subcode:0x0 func:megasas_transition_to_ready
> > [  106.695186] megaraid_sas 0000:03:00.0: System Register set:
> > [  106.889787] megaraid_sas 0000:03:00.0: Failed to transition controller to
> ready for scsi0.
> > [  106.910475] megaraid_sas 0000:03:00.0: Failed from megasas_init_fw
> 6407
> > estuary:/$
> >
> > With the series, now the kernel has direct mapping for the dev as
> > below,
> >
> > estuary:/$ cat /sys/kernel/iommu_groups/0/reserved_regions
> > 0x0000000008000000 0x00000000080fffff msi
> > 0x0000000027b00000 0x00000000286fffff direct
> > 0x00000000e6400000 0x00000000e64fffff direct
> > estuary:/$
> >
> > ....
> > [   12.254318] megaraid_sas 0000:03:00.0: megasas_disable_intr_fusion is
> called outbound_intr_mask:0x40000009
> > [   12.739089] megaraid_sas 0000:03:00.0: FW provided supportMaxExtLDs:
> 0      max_lds: 32
> > [   12.746628] megaraid_sas 0000:03:00.0: controller type       :
> iMR(0MB)
> > [   12.752694] megaraid_sas 0000:03:00.0: Online Controller Reset(OCR)  :
> Enabled
> > [   12.759798] megaraid_sas 0000:03:00.0: Secure JBOD support   : Yes
> > [   12.765778] megaraid_sas 0000:03:00.0: NVMe passthru support : Yes
> > [   12.771931] megaraid_sas 0000:03:00.0: FW provided TM
> TaskAbort/Reset timeou: 6 secs/60 secs
> > [   12.780503] megaraid_sas 0000:03:00.0: JBOD sequence map
> support     : Yes
> > [   12.787000] megaraid_sas 0000:03:00.0: PCI Lane Margining
> support    : No
> > [   12.819179] megaraid_sas 0000:03:00.0: NVME page size        :
> (4096)
> > [   12.825672] megaraid_sas 0000:03:00.0: megasas_enable_intr_fusion is
> called outbound_intr_mask:0x40000000
> > [   12.835199] megaraid_sas 0000:03:00.0: INIT adapter done
> > [   12.873932] megaraid_sas 0000:03:00.0: pci id                :
> (0x1000)/(0x0017)/(0x19e5)/(0xd213)
> > [   12.881644] megaraid_sas 0000:03:00.0: unevenspan support    : no
> > [   12.887451] megaraid_sas 0000:03:00.0: firmware crash dump   : no
> > [   12.893344] megaraid_sas 0000:03:00.0: JBOD sequence map     :
> enabled
> >
> > RAID controller init is now success and can detect the drives
> > attached as well.
> >
> > Thanks,
> > Shameer
> >
> > [0]. https://developer.arm.com/documentation/den0049/latest/
> > [1]. https://github.com/acpica/acpica/pull/638
> >
> > Shameer Kolothum (8):
> >   ACPICA: IORT: Update for revision E
> >   ACPI/IORT: Add support for RMR node parsing
> >   iommu/dma: Introduce generic helper to retrieve RMR info
> >   ACPI/IORT: Add RMR memory regions reservation helper
> >   iommu/arm-smmu-v3: Introduce strtab init helper
> >   iommu/arm-smmu-v3: Add bypass flag to arm_smmu_write_strtab_ent()
> >   iommu/arm-smmu-v3: Get associated RMR info and install bypass STE
> >   iommu/arm-smmu-v3: Reserve any RMR regions associated with a dev
> >
> >  drivers/acpi/arm64/iort.c                   | 182
> +++++++++++++++++++-
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 112 ++++++++++--
> >  drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h |   2 +
> >  drivers/iommu/dma-iommu.c                   |  39 +++++
> >  include/acpi/actbl2.h                       |  25 ++-
> >  include/linux/acpi_iort.h                   |   6 +
> >  include/linux/dma-iommu.h                   |   7 +
> >  include/linux/iommu.h                       |  16 ++
> >  8 files changed, 367 insertions(+), 22 deletions(-)
> >