[PATCH v3 0/2] iommu/smmu-v3: Workaround for hisilicon 161010801 erratum(reserve HW MSI)

[PATCH v3 0/2] iommu/smmu-v3: Workaround for hisilicon 161010801 erratum(reserve HW MSI)

[shameer]

On certain HiSilicon platforms (Hip06/Hip07) the GIC ITS and
PCIe RC deviates from the standard implementation and this breaks
PCIe MSI functionality when SMMU is enabled.

The HiSilicon erratum 161010801 describes this limitation of certain
HiSilicon platforms to support the SMMU mappings for MSI transactions.
On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

To implement this quirk, the following changes are incorporated:

1. Added a generic helper function to IORT code to retrieve and
   reserve the HW ITS address regions.
2. Added quirk to SMMUv3 to reserve HW ITS address regions based
   on IORT SMMUv3 model.

This is based on the following patches:
1. https://patchwork.kernel.org/patch/9740733/
2. https://patchwork.kernel.org/patch/9730491/

Thanks,
Shameer

Changelog:

v2 --> v3
Addressed comments from Lorenzo and Robin:
-Removed dev_is_pci() check in smmuV3 driver.
-Don't treat device not having an ITS mapping as an error in
 iort helper function.

v1 --> v2
-patch 2/2: Invoke iort helper fn based on fwnode type(acpi).

RFCv2 -->PATCH
-Incorporated Lorenzo's review comments.

RFC v1 --> RFC v2
Based on Robin's review comments,
-Removed  the generic erratum framework.
-Using IORT/MADT tables to retrieve the ITS base addr instead
 of vendor specific CSRT table.

shameer (2):
  acpi:iort: Add an IORT helper function to reserve HW ITS address
    regions for IOMMU drivers
  iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/iommu/arm-smmu-v3.c      | 30 +++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 4 files changed, 122 insertions(+), 9 deletions(-)

-- 
1.9.1

[PATCH v3 0/2] iommu/smmu-v3: Workaround for hisilicon 161010801 erratum(reserve HW MSI)

[shameer]

On certain HiSilicon platforms (Hip06/Hip07) the GIC ITS and
PCIe RC deviates from the standard implementation and this breaks
PCIe MSI functionality when SMMU is enabled.

The HiSilicon erratum 161010801 describes this limitation of certain
HiSilicon platforms to support the SMMU mappings for MSI transactions.
On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

To implement this quirk, the following changes are incorporated:

1. Added a generic helper function to IORT code to retrieve and
   reserve the HW ITS address regions.
2. Added quirk to SMMUv3 to reserve HW ITS address regions based
   on IORT SMMUv3 model.

This is based on the following patches:
1. https://patchwork.kernel.org/patch/9740733/
2. https://patchwork.kernel.org/patch/9730491/

Thanks,
Shameer

Changelog:

v2 --> v3
Addressed comments from Lorenzo and Robin:
-Removed dev_is_pci() check in smmuV3 driver.
-Don't treat device not having an ITS mapping as an error in
 iort helper function.

v1 --> v2
-patch 2/2: Invoke iort helper fn based on fwnode type(acpi).

RFCv2 -->PATCH
-Incorporated Lorenzo's review comments.

RFC v1 --> RFC v2
Based on Robin's review comments,
-Removed  the generic erratum framework.
-Using IORT/MADT tables to retrieve the ITS base addr instead
 of vendor specific CSRT table.

shameer (2):
  acpi:iort: Add an IORT helper function to reserve HW ITS address
    regions for IOMMU drivers
  iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/iommu/arm-smmu-v3.c      | 30 +++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 4 files changed, 122 insertions(+), 9 deletions(-)

-- 
1.9.1

[Devel] [PATCH v3 0/2] iommu/smmu-v3: Workaround for hisilicon 161010801 erratum(reserve HW MSI)

[shameer]

[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]

On certain HiSilicon platforms (Hip06/Hip07) the GIC ITS and
PCIe RC deviates from the standard implementation and this breaks
PCIe MSI functionality when SMMU is enabled.

The HiSilicon erratum 161010801 describes this limitation of certain
HiSilicon platforms to support the SMMU mappings for MSI transactions.
On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

To implement this quirk, the following changes are incorporated:

1. Added a generic helper function to IORT code to retrieve and
   reserve the HW ITS address regions.
2. Added quirk to SMMUv3 to reserve HW ITS address regions based
   on IORT SMMUv3 model.

This is based on the following patches:
1. https://patchwork.kernel.org/patch/9740733/
2. https://patchwork.kernel.org/patch/9730491/

Thanks,
Shameer

Changelog:

v2 --> v3
Addressed comments from Lorenzo and Robin:
-Removed dev_is_pci() check in smmuV3 driver.
-Don't treat device not having an ITS mapping as an error in
 iort helper function.

v1 --> v2
-patch 2/2: Invoke iort helper fn based on fwnode type(acpi).

RFCv2 -->PATCH
-Incorporated Lorenzo's review comments.

RFC v1 --> RFC v2
Based on Robin's review comments,
-Removed  the generic erratum framework.
-Using IORT/MADT tables to retrieve the ITS base addr instead
 of vendor specific CSRT table.

shameer (2):
  acpi:iort: Add an IORT helper function to reserve HW ITS address
    regions for IOMMU drivers
  iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/iommu/arm-smmu-v3.c      | 30 +++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 4 files changed, 122 insertions(+), 9 deletions(-)

-- 
1.9.1

[PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[shameer]

The helper function retrieves ITS address regions through IORT
device <-> ITS mappings and reserves it so that these regions
will not be translated by IOMMU and will be excluded from IOVA
allocations. IOMMU drivers can use this to implement their
.get_resv_regions callback.

Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 3 files changed, 96 insertions(+), 5 deletions(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index c5fecf9..cf0a6b8 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -34,6 +34,7 @@
 struct iort_its_msi_chip {
 	struct list_head	list;
 	struct fwnode_handle	*fw_node;
+	phys_addr_t		base_addr;
 	u32			translation_id;
 };
 
@@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
 static DEFINE_SPINLOCK(iort_msi_chip_lock);
 
 /**
- * iort_register_domain_token() - register domain token and related ITS ID
- * to the list from where we can get it back later on.
+ * iort_register_domain_token() - register domain token along with related
+ * ITS ID and base address to the list from where we can get it back later on.
  * @trans_id: ITS ID.
+ * @base: ITS base address.
  * @fw_node: Domain token.
  *
  * Returns: 0 on success, -ENOMEM if no memory when allocating list element
  */
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node)
 {
 	struct iort_its_msi_chip *its_msi_chip;
 
@@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
 
 	its_msi_chip->fw_node = fw_node;
 	its_msi_chip->translation_id = trans_id;
+	its_msi_chip->base_addr = base;
 
 	spin_lock(&iort_msi_chip_lock);
 	list_add(&its_msi_chip->list, &iort_msi_chip_list);
@@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
 	return -ENODEV;
 }
 
+static int iort_find_its_base(u32 its_id, phys_addr_t *base)
+{
+	struct iort_its_msi_chip *its_msi_chip;
+	bool match = false;
+
+	spin_lock(&iort_msi_chip_lock);
+	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
+		if (its_msi_chip->translation_id == its_id) {
+			*base = its_msi_chip->base_addr;
+			match = true;
+			break;
+		}
+	}
+	spin_unlock(&iort_msi_chip_lock);
+
+	return match ? 0 : -ENODEV;
+}
+
 /**
  * iort_dev_find_its_id() - Find the ITS identifier for a device
  * @dev: The device.
@@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 
 	return err;
 }
+
+/**
+ * iort_iommu_its_get_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * Returns: Number of reserved regions on success(0 if no associated ITS),
+ *          appropriate error value otherwise.
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions callback
+ * for reserving the HW ITS address regions.
+ */
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{
+	int i;
+	struct acpi_iort_its_group *its;
+	struct acpi_iort_node *node, *its_node = NULL;
+	int resv = 0;
+
+	node = iort_find_dev_node(dev);
+	if (!node)
+		return -ENODEV;
+
+	if (dev_is_pci(dev)) {
+		u32 rid;
+
+		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
+		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
+	} else {
+		for (i = 0; i < node->mapping_count; i++) {
+			its_node = iort_node_map_platform_id(node, NULL,
+							 IORT_MSI_TYPE, i);
+			if (its_node)
+				break;
+		}
+	}
+
+	if (!its_node)
+		return 0;
+
+	/* Move to ITS specific data */
+	its = (struct acpi_iort_its_group *)its_node->node_data;
+
+	for (i = 0; i < its->its_count; i++) {
+		phys_addr_t base;
+
+		if (!iort_find_its_base(its->identifiers[i], &base)) {
+			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+			struct iommu_resv_region *region;
+
+			region = iommu_alloc_resv_region(base, SZ_128K, prot,
+							 IOMMU_RESV_MSI);
+			if (region) {
+				list_add_tail(&region->list, head);
+				resv++;
+			}
+		}
+	}
+
+	return resv ? : -ENODEV;
+}
 #else
 static inline
 const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
@@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
 static inline
 int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 { return 0; }
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 45ea1933..c45a2ad 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
 		return -ENOMEM;
 	}
 
-	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
+	err = iort_register_domain_token(its_entry->translation_id, res.start,
+					 dom_handle);
 	if (err) {
 		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
 		       &res.start, its_entry->translation_id);
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 3ff9ace..35cf45c 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -26,7 +26,8 @@
 #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
 #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
 
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node);
 void iort_deregister_domain_token(int trans_id);
 struct fwnode_handle *iort_find_domain_token(int trans_id);
 #ifdef CONFIG_ACPI_IORT
@@ -39,6 +40,7 @@
 /* IOMMU interface */
 void iort_set_dma_mask(struct device *dev);
 const struct iommu_ops *iort_iommu_configure(struct device *dev);
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
 #else
 static inline void acpi_iort_init(void) { }
 static inline bool iort_node_match(u8 type) { return false; }
@@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
 static inline
 const struct iommu_ops *iort_iommu_configure(struct device *dev)
 { return NULL; }
+static inline
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
1.9.1

[PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[shameer]

The helper function retrieves ITS address regions through IORT
device <-> ITS mappings and reserves it so that these regions
will not be translated by IOMMU and will be excluded from IOVA
allocations. IOMMU drivers can use this to implement their
.get_resv_regions callback.

Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
---
 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 3 files changed, 96 insertions(+), 5 deletions(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index c5fecf9..cf0a6b8 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -34,6 +34,7 @@
 struct iort_its_msi_chip {
 	struct list_head	list;
 	struct fwnode_handle	*fw_node;
+	phys_addr_t		base_addr;
 	u32			translation_id;
 };
 
@@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
 static DEFINE_SPINLOCK(iort_msi_chip_lock);
 
 /**
- * iort_register_domain_token() - register domain token and related ITS ID
- * to the list from where we can get it back later on.
+ * iort_register_domain_token() - register domain token along with related
+ * ITS ID and base address to the list from where we can get it back later on.
  * @trans_id: ITS ID.
+ * @base: ITS base address.
  * @fw_node: Domain token.
  *
  * Returns: 0 on success, -ENOMEM if no memory when allocating list element
  */
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node)
 {
 	struct iort_its_msi_chip *its_msi_chip;
 
@@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
 
 	its_msi_chip->fw_node = fw_node;
 	its_msi_chip->translation_id = trans_id;
+	its_msi_chip->base_addr = base;
 
 	spin_lock(&iort_msi_chip_lock);
 	list_add(&its_msi_chip->list, &iort_msi_chip_list);
@@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
 	return -ENODEV;
 }
 
+static int iort_find_its_base(u32 its_id, phys_addr_t *base)
+{
+	struct iort_its_msi_chip *its_msi_chip;
+	bool match = false;
+
+	spin_lock(&iort_msi_chip_lock);
+	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
+		if (its_msi_chip->translation_id == its_id) {
+			*base = its_msi_chip->base_addr;
+			match = true;
+			break;
+		}
+	}
+	spin_unlock(&iort_msi_chip_lock);
+
+	return match ? 0 : -ENODEV;
+}
+
 /**
  * iort_dev_find_its_id() - Find the ITS identifier for a device
  * @dev: The device.
@@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 
 	return err;
 }
+
+/**
+ * iort_iommu_its_get_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * Returns: Number of reserved regions on success(0 if no associated ITS),
+ *          appropriate error value otherwise.
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions callback
+ * for reserving the HW ITS address regions.
+ */
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{
+	int i;
+	struct acpi_iort_its_group *its;
+	struct acpi_iort_node *node, *its_node = NULL;
+	int resv = 0;
+
+	node = iort_find_dev_node(dev);
+	if (!node)
+		return -ENODEV;
+
+	if (dev_is_pci(dev)) {
+		u32 rid;
+
+		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
+		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
+	} else {
+		for (i = 0; i < node->mapping_count; i++) {
+			its_node = iort_node_map_platform_id(node, NULL,
+							 IORT_MSI_TYPE, i);
+			if (its_node)
+				break;
+		}
+	}
+
+	if (!its_node)
+		return 0;
+
+	/* Move to ITS specific data */
+	its = (struct acpi_iort_its_group *)its_node->node_data;
+
+	for (i = 0; i < its->its_count; i++) {
+		phys_addr_t base;
+
+		if (!iort_find_its_base(its->identifiers[i], &base)) {
+			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+			struct iommu_resv_region *region;
+
+			region = iommu_alloc_resv_region(base, SZ_128K, prot,
+							 IOMMU_RESV_MSI);
+			if (region) {
+				list_add_tail(&region->list, head);
+				resv++;
+			}
+		}
+	}
+
+	return resv ? : -ENODEV;
+}
 #else
 static inline
 const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
@@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
 static inline
 int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 { return 0; }
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 45ea1933..c45a2ad 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
 		return -ENOMEM;
 	}
 
-	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
+	err = iort_register_domain_token(its_entry->translation_id, res.start,
+					 dom_handle);
 	if (err) {
 		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
 		       &res.start, its_entry->translation_id);
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 3ff9ace..35cf45c 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -26,7 +26,8 @@
 #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
 #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
 
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node);
 void iort_deregister_domain_token(int trans_id);
 struct fwnode_handle *iort_find_domain_token(int trans_id);
 #ifdef CONFIG_ACPI_IORT
@@ -39,6 +40,7 @@
 /* IOMMU interface */
 void iort_set_dma_mask(struct device *dev);
 const struct iommu_ops *iort_iommu_configure(struct device *dev);
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
 #else
 static inline void acpi_iort_init(void) { }
 static inline bool iort_node_match(u8 type) { return false; }
@@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
 static inline
 const struct iommu_ops *iort_iommu_configure(struct device *dev)
 { return NULL; }
+static inline
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
1.9.1

[Devel] [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[shameer]

[-- Attachment #1: Type: text/plain, Size: 6964 bytes --]

The helper function retrieves ITS address regions through IORT
device <-> ITS mappings and reserves it so that these regions
will not be translated by IOMMU and will be excluded from IOVA
allocations. IOMMU drivers can use this to implement their
.get_resv_regions callback.

Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
---
 drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
 drivers/irqchip/irq-gic-v3-its.c |  3 +-
 include/linux/acpi_iort.h        |  7 +++-
 3 files changed, 96 insertions(+), 5 deletions(-)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index c5fecf9..cf0a6b8 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -34,6 +34,7 @@
 struct iort_its_msi_chip {
 	struct list_head	list;
 	struct fwnode_handle	*fw_node;
+	phys_addr_t		base_addr;
 	u32			translation_id;
 };
 
@@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
 static DEFINE_SPINLOCK(iort_msi_chip_lock);
 
 /**
- * iort_register_domain_token() - register domain token and related ITS ID
- * to the list from where we can get it back later on.
+ * iort_register_domain_token() - register domain token along with related
+ * ITS ID and base address to the list from where we can get it back later on.
  * @trans_id: ITS ID.
+ * @base: ITS base address.
  * @fw_node: Domain token.
  *
  * Returns: 0 on success, -ENOMEM if no memory when allocating list element
  */
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node)
 {
 	struct iort_its_msi_chip *its_msi_chip;
 
@@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
 
 	its_msi_chip->fw_node = fw_node;
 	its_msi_chip->translation_id = trans_id;
+	its_msi_chip->base_addr = base;
 
 	spin_lock(&iort_msi_chip_lock);
 	list_add(&its_msi_chip->list, &iort_msi_chip_list);
@@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
 	return -ENODEV;
 }
 
+static int iort_find_its_base(u32 its_id, phys_addr_t *base)
+{
+	struct iort_its_msi_chip *its_msi_chip;
+	bool match = false;
+
+	spin_lock(&iort_msi_chip_lock);
+	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
+		if (its_msi_chip->translation_id == its_id) {
+			*base = its_msi_chip->base_addr;
+			match = true;
+			break;
+		}
+	}
+	spin_unlock(&iort_msi_chip_lock);
+
+	return match ? 0 : -ENODEV;
+}
+
 /**
  * iort_dev_find_its_id() - Find the ITS identifier for a device
  * @dev: The device.
@@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 
 	return err;
 }
+
+/**
+ * iort_iommu_its_get_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * Returns: Number of reserved regions on success(0 if no associated ITS),
+ *          appropriate error value otherwise.
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions callback
+ * for reserving the HW ITS address regions.
+ */
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{
+	int i;
+	struct acpi_iort_its_group *its;
+	struct acpi_iort_node *node, *its_node = NULL;
+	int resv = 0;
+
+	node = iort_find_dev_node(dev);
+	if (!node)
+		return -ENODEV;
+
+	if (dev_is_pci(dev)) {
+		u32 rid;
+
+		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
+		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
+	} else {
+		for (i = 0; i < node->mapping_count; i++) {
+			its_node = iort_node_map_platform_id(node, NULL,
+							 IORT_MSI_TYPE, i);
+			if (its_node)
+				break;
+		}
+	}
+
+	if (!its_node)
+		return 0;
+
+	/* Move to ITS specific data */
+	its = (struct acpi_iort_its_group *)its_node->node_data;
+
+	for (i = 0; i < its->its_count; i++) {
+		phys_addr_t base;
+
+		if (!iort_find_its_base(its->identifiers[i], &base)) {
+			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+			struct iommu_resv_region *region;
+
+			region = iommu_alloc_resv_region(base, SZ_128K, prot,
+							 IOMMU_RESV_MSI);
+			if (region) {
+				list_add_tail(&region->list, head);
+				resv++;
+			}
+		}
+	}
+
+	return resv ? : -ENODEV;
+}
 #else
 static inline
 const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
@@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
 static inline
 int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
 { return 0; }
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
index 45ea1933..c45a2ad 100644
--- a/drivers/irqchip/irq-gic-v3-its.c
+++ b/drivers/irqchip/irq-gic-v3-its.c
@@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
 		return -ENOMEM;
 	}
 
-	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
+	err = iort_register_domain_token(its_entry->translation_id, res.start,
+					 dom_handle);
 	if (err) {
 		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
 		       &res.start, its_entry->translation_id);
diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
index 3ff9ace..35cf45c 100644
--- a/include/linux/acpi_iort.h
+++ b/include/linux/acpi_iort.h
@@ -26,7 +26,8 @@
 #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
 #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
 
-int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
+int iort_register_domain_token(int trans_id, phys_addr_t base,
+			       struct fwnode_handle *fw_node);
 void iort_deregister_domain_token(int trans_id);
 struct fwnode_handle *iort_find_domain_token(int trans_id);
 #ifdef CONFIG_ACPI_IORT
@@ -39,6 +40,7 @@
 /* IOMMU interface */
 void iort_set_dma_mask(struct device *dev);
 const struct iommu_ops *iort_iommu_configure(struct device *dev);
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
 #else
 static inline void acpi_iort_init(void) { }
 static inline bool iort_node_match(u8 type) { return false; }
@@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
 static inline
 const struct iommu_ops *iort_iommu_configure(struct device *dev)
 { return NULL; }
+static inline
+int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
+{ return -ENODEV; }
 #endif
 
 #endif /* __ACPI_IORT_H__ */
-- 
1.9.1

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[shameer]

The HiSilicon erratum 161010801 describes the limitation of HiSilicon
platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.

On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index abe4b88..c9346f2 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -597,6 +597,7 @@ struct arm_smmu_device {
 	u32				features;
 
 #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
+#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
 	u32				options;
 
 	struct arm_smmu_cmdq		cmdq;
@@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
 				      struct list_head *head)
 {
 	struct iommu_resv_region *region;
+	struct arm_smmu_device *smmu;
+	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
 	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+	int resv = 0;
 
-	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
-					 prot, IOMMU_RESV_SW_MSI);
-	if (!region)
+	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
+	if (WARN_ON(!smmu))
 		return;
 
-	list_add_tail(&region->list, head);
+	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
+
+		if (!is_of_node(smmu->dev->fwnode))
+			resv = iort_iommu_its_get_resv_regions(dev, head);
+
+		if (resv < 0) {
+			dev_warn(dev, "HW MSI region resv failed: %d\n", resv);
+			return;
+		}
+	}
+
+	if (!resv) {
+		region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
+						 prot, IOMMU_RESV_SW_MSI);
+		if (!region)
+			return;
+
+		list_add_tail(&region->list, head);
+	}
 
 	iommu_dma_get_resv_regions(dev, head);
 }
@@ -2611,6 +2632,7 @@ static void parse_driver_acpi_options(struct acpi_iort_smmu_v3 *iort_smmu,
 	switch (iort_smmu->model) {
 	case ACPI_IORT_SMMU_HISILICON_HI161X:
 		smmu->options |= ARM_SMMU_OPT_SKIP_PREFETCH;
+		smmu->options |= ARM_SMMU_OPT_RESV_HW_MSI;
 		break;
 	default:
 		break;
-- 
1.9.1

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[shameer]

The HiSilicon erratum 161010801 describes the limitation of HiSilicon
platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.

On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
---
 drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index abe4b88..c9346f2 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -597,6 +597,7 @@ struct arm_smmu_device {
 	u32				features;
 
 #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
+#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
 	u32				options;
 
 	struct arm_smmu_cmdq		cmdq;
@@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
 				      struct list_head *head)
 {
 	struct iommu_resv_region *region;
+	struct arm_smmu_device *smmu;
+	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
 	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+	int resv = 0;
 
-	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
-					 prot, IOMMU_RESV_SW_MSI);
-	if (!region)
+	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
+	if (WARN_ON(!smmu))
 		return;
 
-	list_add_tail(&region->list, head);
+	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
+
+		if (!is_of_node(smmu->dev->fwnode))
+			resv = iort_iommu_its_get_resv_regions(dev, head);
+
+		if (resv < 0) {
+			dev_warn(dev, "HW MSI region resv failed: %d\n", resv);
+			return;
+		}
+	}
+
+	if (!resv) {
+		region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
+						 prot, IOMMU_RESV_SW_MSI);
+		if (!region)
+			return;
+
+		list_add_tail(&region->list, head);
+	}
 
 	iommu_dma_get_resv_regions(dev, head);
 }
@@ -2611,6 +2632,7 @@ static void parse_driver_acpi_options(struct acpi_iort_smmu_v3 *iort_smmu,
 	switch (iort_smmu->model) {
 	case ACPI_IORT_SMMU_HISILICON_HI161X:
 		smmu->options |= ARM_SMMU_OPT_SKIP_PREFETCH;
+		smmu->options |= ARM_SMMU_OPT_RESV_HW_MSI;
 		break;
 	default:
 		break;
-- 
1.9.1

[Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[shameer]

[-- Attachment #1: Type: text/plain, Size: 2704 bytes --]

The HiSilicon erratum 161010801 describes the limitation of HiSilicon
platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.

On these platforms GICv3 ITS translator is presented with the deviceID
by extending the MSI payload data to 64 bits to include the deviceID.
Hence, the PCIe controller on this platforms has to differentiate the
MSI payload against other DMA payload and has to modify the MSI payload.
This basically makes it difficult for this platforms to have a SMMU
translation for MSI.

This patch implements a ACPI table based quirk to reserve the hw msi
regions in the smmu-v3 driver which means these address regions will
not be translated and will be excluded from iova allocations.

Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
---
 drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
 1 file changed, 26 insertions(+), 4 deletions(-)

diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
index abe4b88..c9346f2 100644
--- a/drivers/iommu/arm-smmu-v3.c
+++ b/drivers/iommu/arm-smmu-v3.c
@@ -597,6 +597,7 @@ struct arm_smmu_device {
 	u32				features;
 
 #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
+#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
 	u32				options;
 
 	struct arm_smmu_cmdq		cmdq;
@@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
 				      struct list_head *head)
 {
 	struct iommu_resv_region *region;
+	struct arm_smmu_device *smmu;
+	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
 	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
+	int resv = 0;
 
-	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
-					 prot, IOMMU_RESV_SW_MSI);
-	if (!region)
+	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
+	if (WARN_ON(!smmu))
 		return;
 
-	list_add_tail(&region->list, head);
+	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
+
+		if (!is_of_node(smmu->dev->fwnode))
+			resv = iort_iommu_its_get_resv_regions(dev, head);
+
+		if (resv < 0) {
+			dev_warn(dev, "HW MSI region resv failed: %d\n", resv);
+			return;
+		}
+	}
+
+	if (!resv) {
+		region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
+						 prot, IOMMU_RESV_SW_MSI);
+		if (!region)
+			return;
+
+		list_add_tail(&region->list, head);
+	}
 
 	iommu_dma_get_resv_regions(dev, head);
 }
@@ -2611,6 +2632,7 @@ static void parse_driver_acpi_options(struct acpi_iort_smmu_v3 *iort_smmu,
 	switch (iort_smmu->model) {
 	case ACPI_IORT_SMMU_HISILICON_HI161X:
 		smmu->options |= ARM_SMMU_OPT_SKIP_PREFETCH;
+		smmu->options |= ARM_SMMU_OPT_RESV_HW_MSI;
 		break;
 	default:
 		break;
-- 
1.9.1

Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> The helper function retrieves ITS address regions through IORT
> device <-> ITS mappings and reserves it so that these regions
> will not be translated by IOMMU and will be excluded from IOVA
> allocations. IOMMU drivers can use this to implement their
> .get_resv_regions callback.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
>  drivers/irqchip/irq-gic-v3-its.c |  3 +-
>  include/linux/acpi_iort.h        |  7 +++-
>  3 files changed, 96 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index c5fecf9..cf0a6b8 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -34,6 +34,7 @@
>  struct iort_its_msi_chip {
>  	struct list_head	list;
>  	struct fwnode_handle	*fw_node;
> +	phys_addr_t		base_addr;
>  	u32			translation_id;
>  };
>  
> @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
>  static DEFINE_SPINLOCK(iort_msi_chip_lock);
>  
>  /**
> - * iort_register_domain_token() - register domain token and related ITS ID
> - * to the list from where we can get it back later on.
> + * iort_register_domain_token() - register domain token along with related
> + * ITS ID and base address to the list from where we can get it back later on.
>   * @trans_id: ITS ID.
> + * @base: ITS base address.
>   * @fw_node: Domain token.
>   *
>   * Returns: 0 on success, -ENOMEM if no memory when allocating list element
>   */
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node)
>  {
>  	struct iort_its_msi_chip *its_msi_chip;
>  
> @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
>  
>  	its_msi_chip->fw_node = fw_node;
>  	its_msi_chip->translation_id = trans_id;
> +	its_msi_chip->base_addr = base;
>  
>  	spin_lock(&iort_msi_chip_lock);
>  	list_add(&its_msi_chip->list, &iort_msi_chip_list);
> @@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
>  	return -ENODEV;
>  }
>  
> +static int iort_find_its_base(u32 its_id, phys_addr_t *base)
> +{
> +	struct iort_its_msi_chip *its_msi_chip;
> +	bool match = false;
> +
> +	spin_lock(&iort_msi_chip_lock);
> +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> +		if (its_msi_chip->translation_id == its_id) {
> +			*base = its_msi_chip->base_addr;
> +			match = true;
> +			break;
> +		}
> +	}
> +	spin_unlock(&iort_msi_chip_lock);
> +
> +	return match ? 0 : -ENODEV;
> +}
> +
>  /**
>   * iort_dev_find_its_id() - Find the ITS identifier for a device
>   * @dev: The device.
> @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  
>  	return err;
>  }
> +
> +/**
> + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> + * @dev: Device from iommu_get_resv_regions()
> + * @list: Reserved region list from iommu_get_resv_regions()
> + *
> + * Returns: Number of reserved regions on success(0 if no associated ITS),
> + *          appropriate error value otherwise.
> + *
> + * IOMMU drivers can use this to implement their .get_resv_regions callback
> + * for reserving the HW ITS address regions.
> + */
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{
> +	int i;
> +	struct acpi_iort_its_group *its;
> +	struct acpi_iort_node *node, *its_node = NULL;
> +	int resv = 0;
> +
> +	node = iort_find_dev_node(dev);
> +	if (!node)
> +		return -ENODEV;
> +
> +	if (dev_is_pci(dev)) {
> +		u32 rid;
> +
> +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
> +		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
> +	} else {
> +		for (i = 0; i < node->mapping_count; i++) {
> +			its_node = iort_node_map_platform_id(node, NULL,
> +							 IORT_MSI_TYPE, i);
> +			if (its_node)
> +				break;
> +		}
> +	}
> +
> +	if (!its_node)
> +		return 0;
> +
> +	/* Move to ITS specific data */
> +	its = (struct acpi_iort_its_group *)its_node->node_data;
> +
> +	for (i = 0; i < its->its_count; i++) {
> +		phys_addr_t base;
> +
> +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> +			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +			struct iommu_resv_region *region;
> +
> +			region = iommu_alloc_resv_region(base, SZ_128K, prot,
> +							 IOMMU_RESV_MSI);
> +			if (region) {
> +				list_add_tail(&region->list, head);
> +				resv++;
> +			}
> +		}
> +	}
> +
> +	return resv ? : -ENODEV;

IIUC I think this is not right (ie resv < its_count should count as a
failure).

This should be easy to fix-up but I am not sure there is still time for
these patches to be merged for this merge window, I take the blame since
I am the cause of the delay but I thought the logic in the SMMU driver
should have been clarified and this version does it IMO.

Thanks and apologies,
Lorenzo

> +}
>  #else
>  static inline
>  const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
> @@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
>  static inline
>  int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  { return 0; }
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 45ea1933..c45a2ad 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
>  		return -ENOMEM;
>  	}
>  
> -	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
> +	err = iort_register_domain_token(its_entry->translation_id, res.start,
> +					 dom_handle);
>  	if (err) {
>  		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
>  		       &res.start, its_entry->translation_id);
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index 3ff9ace..35cf45c 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -26,7 +26,8 @@
>  #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
>  #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
>  
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node);
>  void iort_deregister_domain_token(int trans_id);
>  struct fwnode_handle *iort_find_domain_token(int trans_id);
>  #ifdef CONFIG_ACPI_IORT
> @@ -39,6 +40,7 @@
>  /* IOMMU interface */
>  void iort_set_dma_mask(struct device *dev);
>  const struct iommu_ops *iort_iommu_configure(struct device *dev);
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline bool iort_node_match(u8 type) { return false; }
> @@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
>  static inline
>  const struct iommu_ops *iort_iommu_configure(struct device *dev)
>  { return NULL; }
> +static inline
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 1.9.1
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> The helper function retrieves ITS address regions through IORT
> device <-> ITS mappings and reserves it so that these regions
> will not be translated by IOMMU and will be excluded from IOVA
> allocations. IOMMU drivers can use this to implement their
> .get_resv_regions callback.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
>  drivers/irqchip/irq-gic-v3-its.c |  3 +-
>  include/linux/acpi_iort.h        |  7 +++-
>  3 files changed, 96 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index c5fecf9..cf0a6b8 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -34,6 +34,7 @@
>  struct iort_its_msi_chip {
>  	struct list_head	list;
>  	struct fwnode_handle	*fw_node;
> +	phys_addr_t		base_addr;
>  	u32			translation_id;
>  };
>  
> @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
>  static DEFINE_SPINLOCK(iort_msi_chip_lock);
>  
>  /**
> - * iort_register_domain_token() - register domain token and related ITS ID
> - * to the list from where we can get it back later on.
> + * iort_register_domain_token() - register domain token along with related
> + * ITS ID and base address to the list from where we can get it back later on.
>   * @trans_id: ITS ID.
> + * @base: ITS base address.
>   * @fw_node: Domain token.
>   *
>   * Returns: 0 on success, -ENOMEM if no memory when allocating list element
>   */
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node)
>  {
>  	struct iort_its_msi_chip *its_msi_chip;
>  
> @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
>  
>  	its_msi_chip->fw_node = fw_node;
>  	its_msi_chip->translation_id = trans_id;
> +	its_msi_chip->base_addr = base;
>  
>  	spin_lock(&iort_msi_chip_lock);
>  	list_add(&its_msi_chip->list, &iort_msi_chip_list);
> @@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
>  	return -ENODEV;
>  }
>  
> +static int iort_find_its_base(u32 its_id, phys_addr_t *base)
> +{
> +	struct iort_its_msi_chip *its_msi_chip;
> +	bool match = false;
> +
> +	spin_lock(&iort_msi_chip_lock);
> +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> +		if (its_msi_chip->translation_id == its_id) {
> +			*base = its_msi_chip->base_addr;
> +			match = true;
> +			break;
> +		}
> +	}
> +	spin_unlock(&iort_msi_chip_lock);
> +
> +	return match ? 0 : -ENODEV;
> +}
> +
>  /**
>   * iort_dev_find_its_id() - Find the ITS identifier for a device
>   * @dev: The device.
> @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  
>  	return err;
>  }
> +
> +/**
> + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> + * @dev: Device from iommu_get_resv_regions()
> + * @list: Reserved region list from iommu_get_resv_regions()
> + *
> + * Returns: Number of reserved regions on success(0 if no associated ITS),
> + *          appropriate error value otherwise.
> + *
> + * IOMMU drivers can use this to implement their .get_resv_regions callback
> + * for reserving the HW ITS address regions.
> + */
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{
> +	int i;
> +	struct acpi_iort_its_group *its;
> +	struct acpi_iort_node *node, *its_node = NULL;
> +	int resv = 0;
> +
> +	node = iort_find_dev_node(dev);
> +	if (!node)
> +		return -ENODEV;
> +
> +	if (dev_is_pci(dev)) {
> +		u32 rid;
> +
> +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
> +		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
> +	} else {
> +		for (i = 0; i < node->mapping_count; i++) {
> +			its_node = iort_node_map_platform_id(node, NULL,
> +							 IORT_MSI_TYPE, i);
> +			if (its_node)
> +				break;
> +		}
> +	}
> +
> +	if (!its_node)
> +		return 0;
> +
> +	/* Move to ITS specific data */
> +	its = (struct acpi_iort_its_group *)its_node->node_data;
> +
> +	for (i = 0; i < its->its_count; i++) {
> +		phys_addr_t base;
> +
> +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> +			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +			struct iommu_resv_region *region;
> +
> +			region = iommu_alloc_resv_region(base, SZ_128K, prot,
> +							 IOMMU_RESV_MSI);
> +			if (region) {
> +				list_add_tail(&region->list, head);
> +				resv++;
> +			}
> +		}
> +	}
> +
> +	return resv ? : -ENODEV;

IIUC I think this is not right (ie resv < its_count should count as a
failure).

This should be easy to fix-up but I am not sure there is still time for
these patches to be merged for this merge window, I take the blame since
I am the cause of the delay but I thought the logic in the SMMU driver
should have been clarified and this version does it IMO.

Thanks and apologies,
Lorenzo

> +}
>  #else
>  static inline
>  const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
> @@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
>  static inline
>  int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  { return 0; }
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 45ea1933..c45a2ad 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
>  		return -ENOMEM;
>  	}
>  
> -	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
> +	err = iort_register_domain_token(its_entry->translation_id, res.start,
> +					 dom_handle);
>  	if (err) {
>  		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
>  		       &res.start, its_entry->translation_id);
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index 3ff9ace..35cf45c 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -26,7 +26,8 @@
>  #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
>  #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
>  
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node);
>  void iort_deregister_domain_token(int trans_id);
>  struct fwnode_handle *iort_find_domain_token(int trans_id);
>  #ifdef CONFIG_ACPI_IORT
> @@ -39,6 +40,7 @@
>  /* IOMMU interface */
>  void iort_set_dma_mask(struct device *dev);
>  const struct iommu_ops *iort_iommu_configure(struct device *dev);
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline bool iort_node_match(u8 type) { return false; }
> @@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
>  static inline
>  const struct iommu_ops *iort_iommu_configure(struct device *dev)
>  { return NULL; }
> +static inline
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 1.9.1
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Devel] [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

[-- Attachment #1: Type: text/plain, Size: 8039 bytes --]

On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> The helper function retrieves ITS address regions through IORT
> device <-> ITS mappings and reserves it so that these regions
> will not be translated by IOMMU and will be excluded from IOVA
> allocations. IOMMU drivers can use this to implement their
> .get_resv_regions callback.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> ---
>  drivers/acpi/arm64/iort.c        | 91 ++++++++++++++++++++++++++++++++++++++--
>  drivers/irqchip/irq-gic-v3-its.c |  3 +-
>  include/linux/acpi_iort.h        |  7 +++-
>  3 files changed, 96 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> index c5fecf9..cf0a6b8 100644
> --- a/drivers/acpi/arm64/iort.c
> +++ b/drivers/acpi/arm64/iort.c
> @@ -34,6 +34,7 @@
>  struct iort_its_msi_chip {
>  	struct list_head	list;
>  	struct fwnode_handle	*fw_node;
> +	phys_addr_t		base_addr;
>  	u32			translation_id;
>  };
>  
> @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
>  static DEFINE_SPINLOCK(iort_msi_chip_lock);
>  
>  /**
> - * iort_register_domain_token() - register domain token and related ITS ID
> - * to the list from where we can get it back later on.
> + * iort_register_domain_token() - register domain token along with related
> + * ITS ID and base address to the list from where we can get it back later on.
>   * @trans_id: ITS ID.
> + * @base: ITS base address.
>   * @fw_node: Domain token.
>   *
>   * Returns: 0 on success, -ENOMEM if no memory when allocating list element
>   */
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node)
>  {
>  	struct iort_its_msi_chip *its_msi_chip;
>  
> @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node)
>  
>  	its_msi_chip->fw_node = fw_node;
>  	its_msi_chip->translation_id = trans_id;
> +	its_msi_chip->base_addr = base;
>  
>  	spin_lock(&iort_msi_chip_lock);
>  	list_add(&its_msi_chip->list, &iort_msi_chip_list);
> @@ -491,6 +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
>  	return -ENODEV;
>  }
>  
> +static int iort_find_its_base(u32 its_id, phys_addr_t *base)
> +{
> +	struct iort_its_msi_chip *its_msi_chip;
> +	bool match = false;
> +
> +	spin_lock(&iort_msi_chip_lock);
> +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> +		if (its_msi_chip->translation_id == its_id) {
> +			*base = its_msi_chip->base_addr;
> +			match = true;
> +			break;
> +		}
> +	}
> +	spin_unlock(&iort_msi_chip_lock);
> +
> +	return match ? 0 : -ENODEV;
> +}
> +
>  /**
>   * iort_dev_find_its_id() - Find the ITS identifier for a device
>   * @dev: The device.
> @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  
>  	return err;
>  }
> +
> +/**
> + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> + * @dev: Device from iommu_get_resv_regions()
> + * @list: Reserved region list from iommu_get_resv_regions()
> + *
> + * Returns: Number of reserved regions on success(0 if no associated ITS),
> + *          appropriate error value otherwise.
> + *
> + * IOMMU drivers can use this to implement their .get_resv_regions callback
> + * for reserving the HW ITS address regions.
> + */
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{
> +	int i;
> +	struct acpi_iort_its_group *its;
> +	struct acpi_iort_node *node, *its_node = NULL;
> +	int resv = 0;
> +
> +	node = iort_find_dev_node(dev);
> +	if (!node)
> +		return -ENODEV;
> +
> +	if (dev_is_pci(dev)) {
> +		u32 rid;
> +
> +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid, &rid);
> +		its_node = iort_node_map_id(node, rid, NULL, IORT_MSI_TYPE);
> +	} else {
> +		for (i = 0; i < node->mapping_count; i++) {
> +			its_node = iort_node_map_platform_id(node, NULL,
> +							 IORT_MSI_TYPE, i);
> +			if (its_node)
> +				break;
> +		}
> +	}
> +
> +	if (!its_node)
> +		return 0;
> +
> +	/* Move to ITS specific data */
> +	its = (struct acpi_iort_its_group *)its_node->node_data;
> +
> +	for (i = 0; i < its->its_count; i++) {
> +		phys_addr_t base;
> +
> +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> +			int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +			struct iommu_resv_region *region;
> +
> +			region = iommu_alloc_resv_region(base, SZ_128K, prot,
> +							 IOMMU_RESV_MSI);
> +			if (region) {
> +				list_add_tail(&region->list, head);
> +				resv++;
> +			}
> +		}
> +	}
> +
> +	return resv ? : -ENODEV;

IIUC I think this is not right (ie resv < its_count should count as a
failure).

This should be easy to fix-up but I am not sure there is still time for
these patches to be merged for this merge window, I take the blame since
I am the cause of the delay but I thought the logic in the SMMU driver
should have been clarified and this version does it IMO.

Thanks and apologies,
Lorenzo

> +}
>  #else
>  static inline
>  const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
> @@ -656,6 +739,8 @@ const struct iommu_ops *iort_fwspec_iommu_ops(struct iommu_fwspec *fwspec)
>  static inline
>  int iort_add_device_replay(const struct iommu_ops *ops, struct device *dev)
>  { return 0; }
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  static const struct iommu_ops *iort_iommu_xlate(struct device *dev,
> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c
> index 45ea1933..c45a2ad 100644
> --- a/drivers/irqchip/irq-gic-v3-its.c
> +++ b/drivers/irqchip/irq-gic-v3-its.c
> @@ -1854,7 +1854,8 @@ static int __init gic_acpi_parse_madt_its(struct acpi_subtable_header *header,
>  		return -ENOMEM;
>  	}
>  
> -	err = iort_register_domain_token(its_entry->translation_id, dom_handle);
> +	err = iort_register_domain_token(its_entry->translation_id, res.start,
> +					 dom_handle);
>  	if (err) {
>  		pr_err("ITS@%pa: Unable to register GICv3 ITS domain token (ITS ID %d) to IORT\n",
>  		       &res.start, its_entry->translation_id);
> diff --git a/include/linux/acpi_iort.h b/include/linux/acpi_iort.h
> index 3ff9ace..35cf45c 100644
> --- a/include/linux/acpi_iort.h
> +++ b/include/linux/acpi_iort.h
> @@ -26,7 +26,8 @@
>  #define IORT_IRQ_MASK(irq)		(irq & 0xffffffffULL)
>  #define IORT_IRQ_TRIGGER_MASK(irq)	((irq >> 32) & 0xffffffffULL)
>  
> -int iort_register_domain_token(int trans_id, struct fwnode_handle *fw_node);
> +int iort_register_domain_token(int trans_id, phys_addr_t base,
> +			       struct fwnode_handle *fw_node);
>  void iort_deregister_domain_token(int trans_id);
>  struct fwnode_handle *iort_find_domain_token(int trans_id);
>  #ifdef CONFIG_ACPI_IORT
> @@ -39,6 +40,7 @@
>  /* IOMMU interface */
>  void iort_set_dma_mask(struct device *dev);
>  const struct iommu_ops *iort_iommu_configure(struct device *dev);
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head);
>  #else
>  static inline void acpi_iort_init(void) { }
>  static inline bool iort_node_match(u8 type) { return false; }
> @@ -53,6 +55,9 @@ static inline void iort_set_dma_mask(struct device *dev) { }
>  static inline
>  const struct iommu_ops *iort_iommu_configure(struct device *dev)
>  { return NULL; }
> +static inline
> +int iort_iommu_its_get_resv_regions(struct device *dev, struct list_head *head)
> +{ return -ENODEV; }
>  #endif
>  
>  #endif /* __ACPI_IORT_H__ */
> -- 
> 1.9.1
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to majordomo(a)vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi-5wv7dgnIgG8@public.gmane.org]
> Sent: Friday, June 23, 2017 5:54 PM
> To: Shameerali Kolothum Thodi
> Cc: marc.zyngier-5wv7dgnIgG8@public.gmane.org; sudeep.holla-5wv7dgnIgG8@public.gmane.org; will.deacon-5wv7dgnIgG8@public.gmane.org;
> robin.murphy-5wv7dgnIgG8@public.gmane.org; hanjun.guo-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; Gabriele Paoloni; John
> Garry; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-arm-
> kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org; linux-acpi-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; devel-E0kO6a4B6psdnm+yROfE0A@public.gmane.org;
> Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> HW ITS address regions for IOMMU drivers
> 
> On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > The helper function retrieves ITS address regions through IORT device
> > <-> ITS mappings and reserves it so that these regions will not be
> > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > drivers can use this to implement their .get_resv_regions callback.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > ---
> >  drivers/acpi/arm64/iort.c        | 91
> ++++++++++++++++++++++++++++++++++++++--
> >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> >  include/linux/acpi_iort.h        |  7 +++-
> >  3 files changed, 96 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index c5fecf9..cf0a6b8 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -34,6 +34,7 @@
> >  struct iort_its_msi_chip {
> >  	struct list_head	list;
> >  	struct fwnode_handle	*fw_node;
> > +	phys_addr_t		base_addr;
> >  	u32			translation_id;
> >  };
> >
> > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> >
> >  /**
> > - * iort_register_domain_token() - register domain token and related
> > ITS ID
> > - * to the list from where we can get it back later on.
> > + * iort_register_domain_token() - register domain token along with
> > + related
> > + * ITS ID and base address to the list from where we can get it back later
> on.
> >   * @trans_id: ITS ID.
> > + * @base: ITS base address.
> >   * @fw_node: Domain token.
> >   *
> >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> element
> >   */
> > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > *fw_node)
> > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > +			       struct fwnode_handle *fw_node)
> >  {
> >  	struct iort_its_msi_chip *its_msi_chip;
> >
> > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > struct fwnode_handle *fw_node)
> >
> >  	its_msi_chip->fw_node = fw_node;
> >  	its_msi_chip->translation_id = trans_id;
> > +	its_msi_chip->base_addr = base;
> >
> >  	spin_lock(&iort_msi_chip_lock);
> >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> >  	return -ENODEV;
> >  }
> >
> > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > +	struct iort_its_msi_chip *its_msi_chip;
> > +	bool match = false;
> > +
> > +	spin_lock(&iort_msi_chip_lock);
> > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > +		if (its_msi_chip->translation_id == its_id) {
> > +			*base = its_msi_chip->base_addr;
> > +			match = true;
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&iort_msi_chip_lock);
> > +
> > +	return match ? 0 : -ENODEV;
> > +}
> > +
> >  /**
> >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> >   * @dev: The device.
> > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> iommu_ops
> > *ops, struct device *dev)
> >
> >  	return err;
> >  }
> > +
> > +/**
> > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > + * @dev: Device from iommu_get_resv_regions()
> > + * @list: Reserved region list from iommu_get_resv_regions()
> > + *
> > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > + *          appropriate error value otherwise.
> > + *
> > + * IOMMU drivers can use this to implement their .get_resv_regions
> > +callback
> > + * for reserving the HW ITS address regions.
> > + */
> > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > +list_head *head) {
> > +	int i;
> > +	struct acpi_iort_its_group *its;
> > +	struct acpi_iort_node *node, *its_node = NULL;
> > +	int resv = 0;
> > +
> > +	node = iort_find_dev_node(dev);
> > +	if (!node)
> > +		return -ENODEV;
> > +
> > +	if (dev_is_pci(dev)) {
> > +		u32 rid;
> > +
> > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> &rid);
> > +		its_node = iort_node_map_id(node, rid, NULL,
> IORT_MSI_TYPE);
> > +	} else {
> > +		for (i = 0; i < node->mapping_count; i++) {
> > +			its_node = iort_node_map_platform_id(node, NULL,
> > +							 IORT_MSI_TYPE, i);
> > +			if (its_node)
> > +				break;
> > +		}
> > +	}
> > +
> > +	if (!its_node)
> > +		return 0;
> > +
> > +	/* Move to ITS specific data */
> > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > +
> > +	for (i = 0; i < its->its_count; i++) {
> > +		phys_addr_t base;
> > +
> > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> IOMMU_MMIO;
> > +			struct iommu_resv_region *region;
> > +
> > +			region = iommu_alloc_resv_region(base, SZ_128K,
> prot,
> > +							 IOMMU_RESV_MSI);
> > +			if (region) {
> > +				list_add_tail(&region->list, head);
> > +				resv++;
> > +			}
> > +		}
> > +	}
> > +
> > +	return resv ? : -ENODEV;
> 
> IIUC I think this is not right (ie resv < its_count should count as a failure).

Yes, that is a valid error.

> This should be easy to fix-up but I am not sure there is still time for these
> patches to be merged for this merge window, I take the blame since I am the
> cause of the delay but I thought the logic in the SMMU driver should have
> been clarified and this version does it IMO.

No problem and thanks for your support. I could rebase this on  rc1 and send
the next version. Please let me know.

Shameer
 

[PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi at arm.com]
> Sent: Friday, June 23, 2017 5:54 PM
> To: Shameerali Kolothum Thodi
> Cc: marc.zyngier at arm.com; sudeep.holla at arm.com; will.deacon at arm.com;
> robin.murphy at arm.com; hanjun.guo at linaro.org; Gabriele Paoloni; John
> Garry; iommu at lists.linux-foundation.org; linux-arm-
> kernel at lists.infradead.org; linux-acpi at vger.kernel.org; devel at acpica.org;
> Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> HW ITS address regions for IOMMU drivers
> 
> On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > The helper function retrieves ITS address regions through IORT device
> > <-> ITS mappings and reserves it so that these regions will not be
> > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > drivers can use this to implement their .get_resv_regions callback.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  drivers/acpi/arm64/iort.c        | 91
> ++++++++++++++++++++++++++++++++++++++--
> >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> >  include/linux/acpi_iort.h        |  7 +++-
> >  3 files changed, 96 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index c5fecf9..cf0a6b8 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -34,6 +34,7 @@
> >  struct iort_its_msi_chip {
> >  	struct list_head	list;
> >  	struct fwnode_handle	*fw_node;
> > +	phys_addr_t		base_addr;
> >  	u32			translation_id;
> >  };
> >
> > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> >
> >  /**
> > - * iort_register_domain_token() - register domain token and related
> > ITS ID
> > - * to the list from where we can get it back later on.
> > + * iort_register_domain_token() - register domain token along with
> > + related
> > + * ITS ID and base address to the list from where we can get it back later
> on.
> >   * @trans_id: ITS ID.
> > + * @base: ITS base address.
> >   * @fw_node: Domain token.
> >   *
> >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> element
> >   */
> > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > *fw_node)
> > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > +			       struct fwnode_handle *fw_node)
> >  {
> >  	struct iort_its_msi_chip *its_msi_chip;
> >
> > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > struct fwnode_handle *fw_node)
> >
> >  	its_msi_chip->fw_node = fw_node;
> >  	its_msi_chip->translation_id = trans_id;
> > +	its_msi_chip->base_addr = base;
> >
> >  	spin_lock(&iort_msi_chip_lock);
> >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> >  	return -ENODEV;
> >  }
> >
> > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > +	struct iort_its_msi_chip *its_msi_chip;
> > +	bool match = false;
> > +
> > +	spin_lock(&iort_msi_chip_lock);
> > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > +		if (its_msi_chip->translation_id == its_id) {
> > +			*base = its_msi_chip->base_addr;
> > +			match = true;
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&iort_msi_chip_lock);
> > +
> > +	return match ? 0 : -ENODEV;
> > +}
> > +
> >  /**
> >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> >   * @dev: The device.
> > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> iommu_ops
> > *ops, struct device *dev)
> >
> >  	return err;
> >  }
> > +
> > +/**
> > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > + * @dev: Device from iommu_get_resv_regions()
> > + * @list: Reserved region list from iommu_get_resv_regions()
> > + *
> > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > + *          appropriate error value otherwise.
> > + *
> > + * IOMMU drivers can use this to implement their .get_resv_regions
> > +callback
> > + * for reserving the HW ITS address regions.
> > + */
> > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > +list_head *head) {
> > +	int i;
> > +	struct acpi_iort_its_group *its;
> > +	struct acpi_iort_node *node, *its_node = NULL;
> > +	int resv = 0;
> > +
> > +	node = iort_find_dev_node(dev);
> > +	if (!node)
> > +		return -ENODEV;
> > +
> > +	if (dev_is_pci(dev)) {
> > +		u32 rid;
> > +
> > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> &rid);
> > +		its_node = iort_node_map_id(node, rid, NULL,
> IORT_MSI_TYPE);
> > +	} else {
> > +		for (i = 0; i < node->mapping_count; i++) {
> > +			its_node = iort_node_map_platform_id(node, NULL,
> > +							 IORT_MSI_TYPE, i);
> > +			if (its_node)
> > +				break;
> > +		}
> > +	}
> > +
> > +	if (!its_node)
> > +		return 0;
> > +
> > +	/* Move to ITS specific data */
> > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > +
> > +	for (i = 0; i < its->its_count; i++) {
> > +		phys_addr_t base;
> > +
> > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> IOMMU_MMIO;
> > +			struct iommu_resv_region *region;
> > +
> > +			region = iommu_alloc_resv_region(base, SZ_128K,
> prot,
> > +							 IOMMU_RESV_MSI);
> > +			if (region) {
> > +				list_add_tail(&region->list, head);
> > +				resv++;
> > +			}
> > +		}
> > +	}
> > +
> > +	return resv ? : -ENODEV;
> 
> IIUC I think this is not right (ie resv < its_count should count as a failure).

Yes, that is a valid error.

> This should be easy to fix-up but I am not sure there is still time for these
> patches to be merged for this merge window, I take the blame since I am the
> cause of the delay but I thought the logic in the SMMU driver should have
> been clarified and this version does it IMO.

No problem and thanks for your support. I could rebase this on  rc1 and send
the next version. Please let me know.

Shameer
 

Re: [Devel] [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Shameerali Kolothum Thodi]

[-- Attachment #1: Type: text/plain, Size: 6305 bytes --]



> -----Original Message-----
> From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi(a)arm.com]
> Sent: Friday, June 23, 2017 5:54 PM
> To: Shameerali Kolothum Thodi
> Cc: marc.zyngier(a)arm.com; sudeep.holla(a)arm.com; will.deacon(a)arm.com;
> robin.murphy(a)arm.com; hanjun.guo(a)linaro.org; Gabriele Paoloni; John
> Garry; iommu(a)lists.linux-foundation.org; linux-arm-
> kernel(a)lists.infradead.org; linux-acpi(a)vger.kernel.org; devel(a)acpica.org;
> Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> HW ITS address regions for IOMMU drivers
> 
> On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > The helper function retrieves ITS address regions through IORT device
> > <-> ITS mappings and reserves it so that these regions will not be
> > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > drivers can use this to implement their .get_resv_regions callback.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> > ---
> >  drivers/acpi/arm64/iort.c        | 91
> ++++++++++++++++++++++++++++++++++++++--
> >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> >  include/linux/acpi_iort.h        |  7 +++-
> >  3 files changed, 96 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > index c5fecf9..cf0a6b8 100644
> > --- a/drivers/acpi/arm64/iort.c
> > +++ b/drivers/acpi/arm64/iort.c
> > @@ -34,6 +34,7 @@
> >  struct iort_its_msi_chip {
> >  	struct list_head	list;
> >  	struct fwnode_handle	*fw_node;
> > +	phys_addr_t		base_addr;
> >  	u32			translation_id;
> >  };
> >
> > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> >
> >  /**
> > - * iort_register_domain_token() - register domain token and related
> > ITS ID
> > - * to the list from where we can get it back later on.
> > + * iort_register_domain_token() - register domain token along with
> > + related
> > + * ITS ID and base address to the list from where we can get it back later
> on.
> >   * @trans_id: ITS ID.
> > + * @base: ITS base address.
> >   * @fw_node: Domain token.
> >   *
> >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> element
> >   */
> > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > *fw_node)
> > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > +			       struct fwnode_handle *fw_node)
> >  {
> >  	struct iort_its_msi_chip *its_msi_chip;
> >
> > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > struct fwnode_handle *fw_node)
> >
> >  	its_msi_chip->fw_node = fw_node;
> >  	its_msi_chip->translation_id = trans_id;
> > +	its_msi_chip->base_addr = base;
> >
> >  	spin_lock(&iort_msi_chip_lock);
> >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> >  	return -ENODEV;
> >  }
> >
> > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > +	struct iort_its_msi_chip *its_msi_chip;
> > +	bool match = false;
> > +
> > +	spin_lock(&iort_msi_chip_lock);
> > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > +		if (its_msi_chip->translation_id == its_id) {
> > +			*base = its_msi_chip->base_addr;
> > +			match = true;
> > +			break;
> > +		}
> > +	}
> > +	spin_unlock(&iort_msi_chip_lock);
> > +
> > +	return match ? 0 : -ENODEV;
> > +}
> > +
> >  /**
> >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> >   * @dev: The device.
> > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> iommu_ops
> > *ops, struct device *dev)
> >
> >  	return err;
> >  }
> > +
> > +/**
> > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > + * @dev: Device from iommu_get_resv_regions()
> > + * @list: Reserved region list from iommu_get_resv_regions()
> > + *
> > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > + *          appropriate error value otherwise.
> > + *
> > + * IOMMU drivers can use this to implement their .get_resv_regions
> > +callback
> > + * for reserving the HW ITS address regions.
> > + */
> > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > +list_head *head) {
> > +	int i;
> > +	struct acpi_iort_its_group *its;
> > +	struct acpi_iort_node *node, *its_node = NULL;
> > +	int resv = 0;
> > +
> > +	node = iort_find_dev_node(dev);
> > +	if (!node)
> > +		return -ENODEV;
> > +
> > +	if (dev_is_pci(dev)) {
> > +		u32 rid;
> > +
> > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> &rid);
> > +		its_node = iort_node_map_id(node, rid, NULL,
> IORT_MSI_TYPE);
> > +	} else {
> > +		for (i = 0; i < node->mapping_count; i++) {
> > +			its_node = iort_node_map_platform_id(node, NULL,
> > +							 IORT_MSI_TYPE, i);
> > +			if (its_node)
> > +				break;
> > +		}
> > +	}
> > +
> > +	if (!its_node)
> > +		return 0;
> > +
> > +	/* Move to ITS specific data */
> > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > +
> > +	for (i = 0; i < its->its_count; i++) {
> > +		phys_addr_t base;
> > +
> > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> IOMMU_MMIO;
> > +			struct iommu_resv_region *region;
> > +
> > +			region = iommu_alloc_resv_region(base, SZ_128K,
> prot,
> > +							 IOMMU_RESV_MSI);
> > +			if (region) {
> > +				list_add_tail(&region->list, head);
> > +				resv++;
> > +			}
> > +		}
> > +	}
> > +
> > +	return resv ? : -ENODEV;
> 
> IIUC I think this is not right (ie resv < its_count should count as a failure).

Yes, that is a valid error.

> This should be easy to fix-up but I am not sure there is still time for these
> patches to be merged for this merge window, I take the blame since I am the
> cause of the delay but I thought the logic in the SMMU driver should have
> been clarified and this version does it IMO.

No problem and thanks for your support. I could rebase this on  rc1 and send
the next version. Please let me know.

Shameer
 

Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

On Mon, Jul 03, 2017 at 08:18:45AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi@arm.com]
> > Sent: Friday, June 23, 2017 5:54 PM
> > To: Shameerali Kolothum Thodi
> > Cc: marc.zyngier@arm.com; sudeep.holla@arm.com; will.deacon@arm.com;
> > robin.murphy@arm.com; hanjun.guo@linaro.org; Gabriele Paoloni; John
> > Garry; iommu@lists.linux-foundation.org; linux-arm-
> > kernel@lists.infradead.org; linux-acpi@vger.kernel.org; devel@acpica.org;
> > Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> > Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> > HW ITS address regions for IOMMU drivers
> > 
> > On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > > The helper function retrieves ITS address regions through IORT device
> > > <-> ITS mappings and reserves it so that these regions will not be
> > > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > > drivers can use this to implement their .get_resv_regions callback.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >  drivers/acpi/arm64/iort.c        | 91
> > ++++++++++++++++++++++++++++++++++++++--
> > >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> > >  include/linux/acpi_iort.h        |  7 +++-
> > >  3 files changed, 96 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > > index c5fecf9..cf0a6b8 100644
> > > --- a/drivers/acpi/arm64/iort.c
> > > +++ b/drivers/acpi/arm64/iort.c
> > > @@ -34,6 +34,7 @@
> > >  struct iort_its_msi_chip {
> > >  	struct list_head	list;
> > >  	struct fwnode_handle	*fw_node;
> > > +	phys_addr_t		base_addr;
> > >  	u32			translation_id;
> > >  };
> > >
> > > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> > >
> > >  /**
> > > - * iort_register_domain_token() - register domain token and related
> > > ITS ID
> > > - * to the list from where we can get it back later on.
> > > + * iort_register_domain_token() - register domain token along with
> > > + related
> > > + * ITS ID and base address to the list from where we can get it back later
> > on.
> > >   * @trans_id: ITS ID.
> > > + * @base: ITS base address.
> > >   * @fw_node: Domain token.
> > >   *
> > >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> > element
> > >   */
> > > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > > *fw_node)
> > > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > > +			       struct fwnode_handle *fw_node)
> > >  {
> > >  	struct iort_its_msi_chip *its_msi_chip;
> > >
> > > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > > struct fwnode_handle *fw_node)
> > >
> > >  	its_msi_chip->fw_node = fw_node;
> > >  	its_msi_chip->translation_id = trans_id;
> > > +	its_msi_chip->base_addr = base;
> > >
> > >  	spin_lock(&iort_msi_chip_lock);
> > >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> > >  	return -ENODEV;
> > >  }
> > >
> > > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > > +	struct iort_its_msi_chip *its_msi_chip;
> > > +	bool match = false;
> > > +
> > > +	spin_lock(&iort_msi_chip_lock);
> > > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > > +		if (its_msi_chip->translation_id == its_id) {
> > > +			*base = its_msi_chip->base_addr;
> > > +			match = true;
> > > +			break;
> > > +		}
> > > +	}
> > > +	spin_unlock(&iort_msi_chip_lock);
> > > +
> > > +	return match ? 0 : -ENODEV;
> > > +}
> > > +
> > >  /**
> > >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> > >   * @dev: The device.
> > > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> > iommu_ops
> > > *ops, struct device *dev)
> > >
> > >  	return err;
> > >  }
> > > +
> > > +/**
> > > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > > + * @dev: Device from iommu_get_resv_regions()
> > > + * @list: Reserved region list from iommu_get_resv_regions()
> > > + *
> > > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > > + *          appropriate error value otherwise.
> > > + *
> > > + * IOMMU drivers can use this to implement their .get_resv_regions
> > > +callback
> > > + * for reserving the HW ITS address regions.
> > > + */
> > > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > > +list_head *head) {
> > > +	int i;
> > > +	struct acpi_iort_its_group *its;
> > > +	struct acpi_iort_node *node, *its_node = NULL;
> > > +	int resv = 0;
> > > +
> > > +	node = iort_find_dev_node(dev);
> > > +	if (!node)
> > > +		return -ENODEV;
> > > +
> > > +	if (dev_is_pci(dev)) {
> > > +		u32 rid;
> > > +
> > > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> > &rid);
> > > +		its_node = iort_node_map_id(node, rid, NULL,
> > IORT_MSI_TYPE);
> > > +	} else {
> > > +		for (i = 0; i < node->mapping_count; i++) {
> > > +			its_node = iort_node_map_platform_id(node, NULL,
> > > +							 IORT_MSI_TYPE, i);
> > > +			if (its_node)
> > > +				break;
> > > +		}
> > > +	}
> > > +
> > > +	if (!its_node)
> > > +		return 0;
> > > +
> > > +	/* Move to ITS specific data */
> > > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > > +
> > > +	for (i = 0; i < its->its_count; i++) {
> > > +		phys_addr_t base;
> > > +
> > > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> > IOMMU_MMIO;
> > > +			struct iommu_resv_region *region;
> > > +
> > > +			region = iommu_alloc_resv_region(base, SZ_128K,
> > prot,
> > > +							 IOMMU_RESV_MSI);
> > > +			if (region) {
> > > +				list_add_tail(&region->list, head);
> > > +				resv++;
> > > +			}
> > > +		}
> > > +	}
> > > +
> > > +	return resv ? : -ENODEV;
> > 
> > IIUC I think this is not right (ie resv < its_count should count as a failure).
> 
> Yes, that is a valid error.
> 
> > This should be easy to fix-up but I am not sure there is still time for these
> > patches to be merged for this merge window, I take the blame since I am the
> > cause of the delay but I thought the logic in the SMMU driver should have
> > been clarified and this version does it IMO.
> 
> No problem and thanks for your support. I could rebase this on  rc1 and send
> the next version. Please let me know.

Yes, that will work, please resend the final version at -rc1 and if
Will and Robin are ok with SMMU changes we can queue it for 4.14.

Thanks,
Lorenzo

[PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

On Mon, Jul 03, 2017 at 08:18:45AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi at arm.com]
> > Sent: Friday, June 23, 2017 5:54 PM
> > To: Shameerali Kolothum Thodi
> > Cc: marc.zyngier at arm.com; sudeep.holla at arm.com; will.deacon at arm.com;
> > robin.murphy at arm.com; hanjun.guo at linaro.org; Gabriele Paoloni; John
> > Garry; iommu at lists.linux-foundation.org; linux-arm-
> > kernel at lists.infradead.org; linux-acpi at vger.kernel.org; devel at acpica.org;
> > Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> > Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> > HW ITS address regions for IOMMU drivers
> > 
> > On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > > The helper function retrieves ITS address regions through IORT device
> > > <-> ITS mappings and reserves it so that these regions will not be
> > > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > > drivers can use this to implement their .get_resv_regions callback.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >  drivers/acpi/arm64/iort.c        | 91
> > ++++++++++++++++++++++++++++++++++++++--
> > >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> > >  include/linux/acpi_iort.h        |  7 +++-
> > >  3 files changed, 96 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > > index c5fecf9..cf0a6b8 100644
> > > --- a/drivers/acpi/arm64/iort.c
> > > +++ b/drivers/acpi/arm64/iort.c
> > > @@ -34,6 +34,7 @@
> > >  struct iort_its_msi_chip {
> > >  	struct list_head	list;
> > >  	struct fwnode_handle	*fw_node;
> > > +	phys_addr_t		base_addr;
> > >  	u32			translation_id;
> > >  };
> > >
> > > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> > >
> > >  /**
> > > - * iort_register_domain_token() - register domain token and related
> > > ITS ID
> > > - * to the list from where we can get it back later on.
> > > + * iort_register_domain_token() - register domain token along with
> > > + related
> > > + * ITS ID and base address to the list from where we can get it back later
> > on.
> > >   * @trans_id: ITS ID.
> > > + * @base: ITS base address.
> > >   * @fw_node: Domain token.
> > >   *
> > >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> > element
> > >   */
> > > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > > *fw_node)
> > > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > > +			       struct fwnode_handle *fw_node)
> > >  {
> > >  	struct iort_its_msi_chip *its_msi_chip;
> > >
> > > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > > struct fwnode_handle *fw_node)
> > >
> > >  	its_msi_chip->fw_node = fw_node;
> > >  	its_msi_chip->translation_id = trans_id;
> > > +	its_msi_chip->base_addr = base;
> > >
> > >  	spin_lock(&iort_msi_chip_lock);
> > >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> > >  	return -ENODEV;
> > >  }
> > >
> > > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > > +	struct iort_its_msi_chip *its_msi_chip;
> > > +	bool match = false;
> > > +
> > > +	spin_lock(&iort_msi_chip_lock);
> > > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > > +		if (its_msi_chip->translation_id == its_id) {
> > > +			*base = its_msi_chip->base_addr;
> > > +			match = true;
> > > +			break;
> > > +		}
> > > +	}
> > > +	spin_unlock(&iort_msi_chip_lock);
> > > +
> > > +	return match ? 0 : -ENODEV;
> > > +}
> > > +
> > >  /**
> > >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> > >   * @dev: The device.
> > > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> > iommu_ops
> > > *ops, struct device *dev)
> > >
> > >  	return err;
> > >  }
> > > +
> > > +/**
> > > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > > + * @dev: Device from iommu_get_resv_regions()
> > > + * @list: Reserved region list from iommu_get_resv_regions()
> > > + *
> > > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > > + *          appropriate error value otherwise.
> > > + *
> > > + * IOMMU drivers can use this to implement their .get_resv_regions
> > > +callback
> > > + * for reserving the HW ITS address regions.
> > > + */
> > > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > > +list_head *head) {
> > > +	int i;
> > > +	struct acpi_iort_its_group *its;
> > > +	struct acpi_iort_node *node, *its_node = NULL;
> > > +	int resv = 0;
> > > +
> > > +	node = iort_find_dev_node(dev);
> > > +	if (!node)
> > > +		return -ENODEV;
> > > +
> > > +	if (dev_is_pci(dev)) {
> > > +		u32 rid;
> > > +
> > > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> > &rid);
> > > +		its_node = iort_node_map_id(node, rid, NULL,
> > IORT_MSI_TYPE);
> > > +	} else {
> > > +		for (i = 0; i < node->mapping_count; i++) {
> > > +			its_node = iort_node_map_platform_id(node, NULL,
> > > +							 IORT_MSI_TYPE, i);
> > > +			if (its_node)
> > > +				break;
> > > +		}
> > > +	}
> > > +
> > > +	if (!its_node)
> > > +		return 0;
> > > +
> > > +	/* Move to ITS specific data */
> > > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > > +
> > > +	for (i = 0; i < its->its_count; i++) {
> > > +		phys_addr_t base;
> > > +
> > > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> > IOMMU_MMIO;
> > > +			struct iommu_resv_region *region;
> > > +
> > > +			region = iommu_alloc_resv_region(base, SZ_128K,
> > prot,
> > > +							 IOMMU_RESV_MSI);
> > > +			if (region) {
> > > +				list_add_tail(&region->list, head);
> > > +				resv++;
> > > +			}
> > > +		}
> > > +	}
> > > +
> > > +	return resv ? : -ENODEV;
> > 
> > IIUC I think this is not right (ie resv < its_count should count as a failure).
> 
> Yes, that is a valid error.
> 
> > This should be easy to fix-up but I am not sure there is still time for these
> > patches to be merged for this merge window, I take the blame since I am the
> > cause of the delay but I thought the logic in the SMMU driver should have
> > been clarified and this version does it IMO.
> 
> No problem and thanks for your support. I could rebase this on  rc1 and send
> the next version. Please let me know.

Yes, that will work, please resend the final version at -rc1 and if
Will and Robin are ok with SMMU changes we can queue it for 4.14.

Thanks,
Lorenzo

Re: [Devel] [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve HW ITS address regions for IOMMU drivers

[Lorenzo Pieralisi]

[-- Attachment #1: Type: text/plain, Size: 6887 bytes --]

On Mon, Jul 03, 2017 at 08:18:45AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Lorenzo Pieralisi [mailto:lorenzo.pieralisi(a)arm.com]
> > Sent: Friday, June 23, 2017 5:54 PM
> > To: Shameerali Kolothum Thodi
> > Cc: marc.zyngier(a)arm.com; sudeep.holla(a)arm.com; will.deacon(a)arm.com;
> > robin.murphy(a)arm.com; hanjun.guo(a)linaro.org; Gabriele Paoloni; John
> > Garry; iommu(a)lists.linux-foundation.org; linux-arm-
> > kernel(a)lists.infradead.org; linux-acpi(a)vger.kernel.org; devel(a)acpica.org;
> > Linuxarm; Wangzhou (B); Guohanjun (Hanjun Guo)
> > Subject: Re: [PATCH v3 1/2] acpi:iort: Add an IORT helper function to reserve
> > HW ITS address regions for IOMMU drivers
> > 
> > On Fri, Jun 23, 2017 at 03:58:00PM +0100, shameer wrote:
> > > The helper function retrieves ITS address regions through IORT device
> > > <-> ITS mappings and reserves it so that these regions will not be
> > > translated by IOMMU and will be excluded from IOVA allocations. IOMMU
> > > drivers can use this to implement their .get_resv_regions callback.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> > > ---
> > >  drivers/acpi/arm64/iort.c        | 91
> > ++++++++++++++++++++++++++++++++++++++--
> > >  drivers/irqchip/irq-gic-v3-its.c |  3 +-
> > >  include/linux/acpi_iort.h        |  7 +++-
> > >  3 files changed, 96 insertions(+), 5 deletions(-)
> > >
> > > diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
> > > index c5fecf9..cf0a6b8 100644
> > > --- a/drivers/acpi/arm64/iort.c
> > > +++ b/drivers/acpi/arm64/iort.c
> > > @@ -34,6 +34,7 @@
> > >  struct iort_its_msi_chip {
> > >  	struct list_head	list;
> > >  	struct fwnode_handle	*fw_node;
> > > +	phys_addr_t		base_addr;
> > >  	u32			translation_id;
> > >  };
> > >
> > > @@ -131,14 +132,16 @@ typedef acpi_status (*iort_find_node_callback)
> > > static DEFINE_SPINLOCK(iort_msi_chip_lock);
> > >
> > >  /**
> > > - * iort_register_domain_token() - register domain token and related
> > > ITS ID
> > > - * to the list from where we can get it back later on.
> > > + * iort_register_domain_token() - register domain token along with
> > > + related
> > > + * ITS ID and base address to the list from where we can get it back later
> > on.
> > >   * @trans_id: ITS ID.
> > > + * @base: ITS base address.
> > >   * @fw_node: Domain token.
> > >   *
> > >   * Returns: 0 on success, -ENOMEM if no memory when allocating list
> > element
> > >   */
> > > -int iort_register_domain_token(int trans_id, struct fwnode_handle
> > > *fw_node)
> > > +int iort_register_domain_token(int trans_id, phys_addr_t base,
> > > +			       struct fwnode_handle *fw_node)
> > >  {
> > >  	struct iort_its_msi_chip *its_msi_chip;
> > >
> > > @@ -148,6 +151,7 @@ int iort_register_domain_token(int trans_id,
> > > struct fwnode_handle *fw_node)
> > >
> > >  	its_msi_chip->fw_node = fw_node;
> > >  	its_msi_chip->translation_id = trans_id;
> > > +	its_msi_chip->base_addr = base;
> > >
> > >  	spin_lock(&iort_msi_chip_lock);
> > >  	list_add(&its_msi_chip->list, &iort_msi_chip_list); @@ -491,6
> > > +495,24 @@ int iort_pmsi_get_dev_id(struct device *dev, u32 *dev_id)
> > >  	return -ENODEV;
> > >  }
> > >
> > > +static int iort_find_its_base(u32 its_id, phys_addr_t *base) {
> > > +	struct iort_its_msi_chip *its_msi_chip;
> > > +	bool match = false;
> > > +
> > > +	spin_lock(&iort_msi_chip_lock);
> > > +	list_for_each_entry(its_msi_chip, &iort_msi_chip_list, list) {
> > > +		if (its_msi_chip->translation_id == its_id) {
> > > +			*base = its_msi_chip->base_addr;
> > > +			match = true;
> > > +			break;
> > > +		}
> > > +	}
> > > +	spin_unlock(&iort_msi_chip_lock);
> > > +
> > > +	return match ? 0 : -ENODEV;
> > > +}
> > > +
> > >  /**
> > >   * iort_dev_find_its_id() - Find the ITS identifier for a device
> > >   * @dev: The device.
> > > @@ -649,6 +671,67 @@ int iort_add_device_replay(const struct
> > iommu_ops
> > > *ops, struct device *dev)
> > >
> > >  	return err;
> > >  }
> > > +
> > > +/**
> > > + * iort_iommu_its_get_resv_regions - Reserved region driver helper
> > > + * @dev: Device from iommu_get_resv_regions()
> > > + * @list: Reserved region list from iommu_get_resv_regions()
> > > + *
> > > + * Returns: Number of reserved regions on success(0 if no associated ITS),
> > > + *          appropriate error value otherwise.
> > > + *
> > > + * IOMMU drivers can use this to implement their .get_resv_regions
> > > +callback
> > > + * for reserving the HW ITS address regions.
> > > + */
> > > +int iort_iommu_its_get_resv_regions(struct device *dev, struct
> > > +list_head *head) {
> > > +	int i;
> > > +	struct acpi_iort_its_group *its;
> > > +	struct acpi_iort_node *node, *its_node = NULL;
> > > +	int resv = 0;
> > > +
> > > +	node = iort_find_dev_node(dev);
> > > +	if (!node)
> > > +		return -ENODEV;
> > > +
> > > +	if (dev_is_pci(dev)) {
> > > +		u32 rid;
> > > +
> > > +		pci_for_each_dma_alias(to_pci_dev(dev), __get_pci_rid,
> > &rid);
> > > +		its_node = iort_node_map_id(node, rid, NULL,
> > IORT_MSI_TYPE);
> > > +	} else {
> > > +		for (i = 0; i < node->mapping_count; i++) {
> > > +			its_node = iort_node_map_platform_id(node, NULL,
> > > +							 IORT_MSI_TYPE, i);
> > > +			if (its_node)
> > > +				break;
> > > +		}
> > > +	}
> > > +
> > > +	if (!its_node)
> > > +		return 0;
> > > +
> > > +	/* Move to ITS specific data */
> > > +	its = (struct acpi_iort_its_group *)its_node->node_data;
> > > +
> > > +	for (i = 0; i < its->its_count; i++) {
> > > +		phys_addr_t base;
> > > +
> > > +		if (!iort_find_its_base(its->identifiers[i], &base)) {
> > > +			int prot = IOMMU_WRITE | IOMMU_NOEXEC |
> > IOMMU_MMIO;
> > > +			struct iommu_resv_region *region;
> > > +
> > > +			region = iommu_alloc_resv_region(base, SZ_128K,
> > prot,
> > > +							 IOMMU_RESV_MSI);
> > > +			if (region) {
> > > +				list_add_tail(&region->list, head);
> > > +				resv++;
> > > +			}
> > > +		}
> > > +	}
> > > +
> > > +	return resv ? : -ENODEV;
> > 
> > IIUC I think this is not right (ie resv < its_count should count as a failure).
> 
> Yes, that is a valid error.
> 
> > This should be easy to fix-up but I am not sure there is still time for these
> > patches to be merged for this merge window, I take the blame since I am the
> > cause of the delay but I thought the logic in the SMMU driver should have
> > been clarified and this version does it IMO.
> 
> No problem and thanks for your support. I could rebase this on  rc1 and send
> the next version. Please let me know.

Yes, that will work, please resend the final version at -rc1 and if
Will and Robin are ok with SMMU changes we can queue it for 4.14.

Thanks,
Lorenzo

Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.
> 
> On these platforms GICv3 ITS translator is presented with the deviceID
> by extending the MSI payload data to 64 bits to include the deviceID.
> Hence, the PCIe controller on this platforms has to differentiate the
> MSI payload against other DMA payload and has to modify the MSI payload.
> This basically makes it difficult for this platforms to have a SMMU
> translation for MSI.
> 
> This patch implements a ACPI table based quirk to reserve the hw msi
> regions in the smmu-v3 driver which means these address regions will
> not be translated and will be excluded from iova allocations.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
>  1 file changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
> index abe4b88..c9346f2 100644
> --- a/drivers/iommu/arm-smmu-v3.c
> +++ b/drivers/iommu/arm-smmu-v3.c
> @@ -597,6 +597,7 @@ struct arm_smmu_device {
>  	u32				features;
>  
>  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
>  	u32				options;
>  
>  	struct arm_smmu_cmdq		cmdq;
> @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
>  				      struct list_head *head)
>  {
>  	struct iommu_resv_region *region;
> +	struct arm_smmu_device *smmu;
> +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
>  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +	int resv = 0;
>  
> -	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
> -					 prot, IOMMU_RESV_SW_MSI);
> -	if (!region)
> +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);

Does this callback actually get called without a prior ->add_device callback
for the master in question? If not, then we can already claw the structure
out via the iommu_priv field in the fwspec.

> +	if (WARN_ON(!smmu))

Again, how does this trigger?

>  		return;
>  
> -	list_add_tail(&region->list, head);
> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> +
> +		if (!is_of_node(smmu->dev->fwnode))
> +			resv = iort_iommu_its_get_resv_regions(dev, head);

How does this work when we're not using ACPI? Shouldn't of vs ACPI be
abstracted from the driver?

Will

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.
> 
> On these platforms GICv3 ITS translator is presented with the deviceID
> by extending the MSI payload data to 64 bits to include the deviceID.
> Hence, the PCIe controller on this platforms has to differentiate the
> MSI payload against other DMA payload and has to modify the MSI payload.
> This basically makes it difficult for this platforms to have a SMMU
> translation for MSI.
> 
> This patch implements a ACPI table based quirk to reserve the hw msi
> regions in the smmu-v3 driver which means these address regions will
> not be translated and will be excluded from iova allocations.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> ---
>  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
>  1 file changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
> index abe4b88..c9346f2 100644
> --- a/drivers/iommu/arm-smmu-v3.c
> +++ b/drivers/iommu/arm-smmu-v3.c
> @@ -597,6 +597,7 @@ struct arm_smmu_device {
>  	u32				features;
>  
>  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
>  	u32				options;
>  
>  	struct arm_smmu_cmdq		cmdq;
> @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
>  				      struct list_head *head)
>  {
>  	struct iommu_resv_region *region;
> +	struct arm_smmu_device *smmu;
> +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
>  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +	int resv = 0;
>  
> -	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
> -					 prot, IOMMU_RESV_SW_MSI);
> -	if (!region)
> +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);

Does this callback actually get called without a prior ->add_device callback
for the master in question? If not, then we can already claw the structure
out via the iommu_priv field in the fwspec.

> +	if (WARN_ON(!smmu))

Again, how does this trigger?

>  		return;
>  
> -	list_add_tail(&region->list, head);
> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> +
> +		if (!is_of_node(smmu->dev->fwnode))
> +			resv = iort_iommu_its_get_resv_regions(dev, head);

How does this work when we're not using ACPI? Shouldn't of vs ACPI be
abstracted from the driver?

Will

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

[-- Attachment #1: Type: text/plain, Size: 2540 bytes --]

On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> platforms Hip06/Hip07 to support the SMMU mappings for MSI transactions.
> 
> On these platforms GICv3 ITS translator is presented with the deviceID
> by extending the MSI payload data to 64 bits to include the deviceID.
> Hence, the PCIe controller on this platforms has to differentiate the
> MSI payload against other DMA payload and has to modify the MSI payload.
> This basically makes it difficult for this platforms to have a SMMU
> translation for MSI.
> 
> This patch implements a ACPI table based quirk to reserve the hw msi
> regions in the smmu-v3 driver which means these address regions will
> not be translated and will be excluded from iova allocations.
> 
> Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> ---
>  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
>  1 file changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c
> index abe4b88..c9346f2 100644
> --- a/drivers/iommu/arm-smmu-v3.c
> +++ b/drivers/iommu/arm-smmu-v3.c
> @@ -597,6 +597,7 @@ struct arm_smmu_device {
>  	u32				features;
>  
>  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
>  	u32				options;
>  
>  	struct arm_smmu_cmdq		cmdq;
> @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct device *dev,
>  				      struct list_head *head)
>  {
>  	struct iommu_resv_region *region;
> +	struct arm_smmu_device *smmu;
> +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
>  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> +	int resv = 0;
>  
> -	region = iommu_alloc_resv_region(MSI_IOVA_BASE, MSI_IOVA_LENGTH,
> -					 prot, IOMMU_RESV_SW_MSI);
> -	if (!region)
> +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);

Does this callback actually get called without a prior ->add_device callback
for the master in question? If not, then we can already claw the structure
out via the iommu_priv field in the fwspec.

> +	if (WARN_ON(!smmu))

Again, how does this trigger?

>  		return;
>  
> -	list_add_tail(&region->list, head);
> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> +
> +		if (!is_of_node(smmu->dev->fwnode))
> +			resv = iort_iommu_its_get_resv_regions(dev, head);

How does this work when we're not using ACPI? Shouldn't of vs ACPI be
abstracted from the driver?

Will

RE: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon@arm.com]
> Sent: Tuesday, July 04, 2017 6:38 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
> sudeep.holla@arm.com; robin.murphy@arm.com; hanjun.guo@linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel@lists.infradead.org; devel@acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> transactions.
> >
> > On these platforms GICv3 ITS translator is presented with the deviceID
> > by extending the MSI payload data to 64 bits to include the deviceID.
> > Hence, the PCIe controller on this platforms has to differentiate the
> > MSI payload against other DMA payload and has to modify the MSI
> payload.
> > This basically makes it difficult for this platforms to have a SMMU
> > translation for MSI.
> >
> > This patch implements a ACPI table based quirk to reserve the hw msi
> > regions in the smmu-v3 driver which means these address regions will
> > not be translated and will be excluded from iova allocations.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> >  1 file changed, 26 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> v3.c
> > index abe4b88..c9346f2 100644
> > --- a/drivers/iommu/arm-smmu-v3.c
> > +++ b/drivers/iommu/arm-smmu-v3.c
> > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> >  	u32				features;
> >
> >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> >  	u32				options;
> >
> >  	struct arm_smmu_cmdq		cmdq;
> > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> device *dev,
> >  				      struct list_head *head)
> >  {
> >  	struct iommu_resv_region *region;
> > +	struct arm_smmu_device *smmu;
> > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > +	int resv = 0;
> >
> > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> MSI_IOVA_LENGTH,
> > -					 prot, IOMMU_RESV_SW_MSI);
> > -	if (!region)
> > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> 
> Does this callback actually get called without a prior ->add_device callback
> for the master in question? If not, then we can already claw the structure
> out via the iommu_priv field in the fwspec.

Thanks Will for going through this.

Yes, from the logs I have, it looks like _resv callback is always called after
 ->add_device. I will double check this with vfio bind case as well.

And I guess, this is what you are proposing to retrieve the smmu,

master = dev->iommu_fwspec->iommu_priv;
smmu = master->smmu;

> > +	if (WARN_ON(!smmu))
>
> Again, how does this trigger?
> >  		return;
> >
> > -	list_add_tail(&region->list, head);
> > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > +
> > +		if (!is_of_node(smmu->dev->fwnode))
> > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> 
> How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> abstracted from the driver?

At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
this is a low priority for us at the moment. Is the suggestion is to have a common function
outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
is the best way here. 

Thanks,
Shameer

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon at arm.com]
> Sent: Tuesday, July 04, 2017 6:38 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
> sudeep.holla at arm.com; robin.murphy at arm.com; hanjun.guo at linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi at vger.kernel.org;
> iommu at lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel at lists.infradead.org; devel at acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> transactions.
> >
> > On these platforms GICv3 ITS translator is presented with the deviceID
> > by extending the MSI payload data to 64 bits to include the deviceID.
> > Hence, the PCIe controller on this platforms has to differentiate the
> > MSI payload against other DMA payload and has to modify the MSI
> payload.
> > This basically makes it difficult for this platforms to have a SMMU
> > translation for MSI.
> >
> > This patch implements a ACPI table based quirk to reserve the hw msi
> > regions in the smmu-v3 driver which means these address regions will
> > not be translated and will be excluded from iova allocations.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > ---
> >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> >  1 file changed, 26 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> v3.c
> > index abe4b88..c9346f2 100644
> > --- a/drivers/iommu/arm-smmu-v3.c
> > +++ b/drivers/iommu/arm-smmu-v3.c
> > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> >  	u32				features;
> >
> >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> >  	u32				options;
> >
> >  	struct arm_smmu_cmdq		cmdq;
> > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> device *dev,
> >  				      struct list_head *head)
> >  {
> >  	struct iommu_resv_region *region;
> > +	struct arm_smmu_device *smmu;
> > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > +	int resv = 0;
> >
> > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> MSI_IOVA_LENGTH,
> > -					 prot, IOMMU_RESV_SW_MSI);
> > -	if (!region)
> > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> 
> Does this callback actually get called without a prior ->add_device callback
> for the master in question? If not, then we can already claw the structure
> out via the iommu_priv field in the fwspec.

Thanks Will for going through this.

Yes, from the logs I have, it looks like _resv callback is always called after
 ->add_device. I will double check this with vfio bind case as well.

And I guess, this is what you are proposing to retrieve the smmu,

master = dev->iommu_fwspec->iommu_priv;
smmu = master->smmu;

> > +	if (WARN_ON(!smmu))
>
> Again, how does this trigger?
> >  		return;
> >
> > -	list_add_tail(&region->list, head);
> > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > +
> > +		if (!is_of_node(smmu->dev->fwnode))
> > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> 
> How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> abstracted from the driver?

At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
this is a low priority for us at the moment. Is the suggestion is to have a common function
outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
is the best way here. 

Thanks,
Shameer

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

[-- Attachment #1: Type: text/plain, Size: 3894 bytes --]



> -----Original Message-----
> From: Will Deacon [mailto:will.deacon(a)arm.com]
> Sent: Tuesday, July 04, 2017 6:38 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
> sudeep.holla(a)arm.com; robin.murphy(a)arm.com; hanjun.guo(a)linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi(a)vger.kernel.org;
> iommu(a)lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel(a)lists.infradead.org; devel(a)acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> transactions.
> >
> > On these platforms GICv3 ITS translator is presented with the deviceID
> > by extending the MSI payload data to 64 bits to include the deviceID.
> > Hence, the PCIe controller on this platforms has to differentiate the
> > MSI payload against other DMA payload and has to modify the MSI
> payload.
> > This basically makes it difficult for this platforms to have a SMMU
> > translation for MSI.
> >
> > This patch implements a ACPI table based quirk to reserve the hw msi
> > regions in the smmu-v3 driver which means these address regions will
> > not be translated and will be excluded from iova allocations.
> >
> > Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> > ---
> >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> >  1 file changed, 26 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> v3.c
> > index abe4b88..c9346f2 100644
> > --- a/drivers/iommu/arm-smmu-v3.c
> > +++ b/drivers/iommu/arm-smmu-v3.c
> > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> >  	u32				features;
> >
> >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> >  	u32				options;
> >
> >  	struct arm_smmu_cmdq		cmdq;
> > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> device *dev,
> >  				      struct list_head *head)
> >  {
> >  	struct iommu_resv_region *region;
> > +	struct arm_smmu_device *smmu;
> > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > +	int resv = 0;
> >
> > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> MSI_IOVA_LENGTH,
> > -					 prot, IOMMU_RESV_SW_MSI);
> > -	if (!region)
> > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> 
> Does this callback actually get called without a prior ->add_device callback
> for the master in question? If not, then we can already claw the structure
> out via the iommu_priv field in the fwspec.

Thanks Will for going through this.

Yes, from the logs I have, it looks like _resv callback is always called after
 ->add_device. I will double check this with vfio bind case as well.

And I guess, this is what you are proposing to retrieve the smmu,

master = dev->iommu_fwspec->iommu_priv;
smmu = master->smmu;

> > +	if (WARN_ON(!smmu))
>
> Again, how does this trigger?
> >  		return;
> >
> > -	list_add_tail(&region->list, head);
> > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > +
> > +		if (!is_of_node(smmu->dev->fwnode))
> > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> 
> How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> abstracted from the driver?

At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
this is a low priority for us at the moment. Is the suggestion is to have a common function
outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
is the best way here. 

Thanks,
Shameer

Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

On Wed, Jul 05, 2017 at 09:48:53AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Will Deacon [mailto:will.deacon@arm.com]
> > Sent: Tuesday, July 04, 2017 6:38 PM
> > To: Shameerali Kolothum Thodi
> > Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
> > sudeep.holla@arm.com; robin.murphy@arm.com; hanjun.guo@linaro.org;
> > Gabriele Paoloni; John Garry; Linuxarm; linux-acpi@vger.kernel.org;
> > iommu@lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> > linux-arm-kernel@lists.infradead.org; devel@acpica.org
> > Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> > HiSilicon erratum 161010801
> > 
> > On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> > transactions.
> > >
> > > On these platforms GICv3 ITS translator is presented with the deviceID
> > > by extending the MSI payload data to 64 bits to include the deviceID.
> > > Hence, the PCIe controller on this platforms has to differentiate the
> > > MSI payload against other DMA payload and has to modify the MSI
> > payload.
> > > This basically makes it difficult for this platforms to have a SMMU
> > > translation for MSI.
> > >
> > > This patch implements a ACPI table based quirk to reserve the hw msi
> > > regions in the smmu-v3 driver which means these address regions will
> > > not be translated and will be excluded from iova allocations.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> > >  1 file changed, 26 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> > v3.c
> > > index abe4b88..c9346f2 100644
> > > --- a/drivers/iommu/arm-smmu-v3.c
> > > +++ b/drivers/iommu/arm-smmu-v3.c
> > > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> > >  	u32				features;
> > >
> > >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> > >  	u32				options;
> > >
> > >  	struct arm_smmu_cmdq		cmdq;
> > > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> > device *dev,
> > >  				      struct list_head *head)
> > >  {
> > >  	struct iommu_resv_region *region;
> > > +	struct arm_smmu_device *smmu;
> > > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> > >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > > +	int resv = 0;
> > >
> > > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> > MSI_IOVA_LENGTH,
> > > -					 prot, IOMMU_RESV_SW_MSI);
> > > -	if (!region)
> > > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> > 
> > Does this callback actually get called without a prior ->add_device callback
> > for the master in question? If not, then we can already claw the structure
> > out via the iommu_priv field in the fwspec.
> 
> Thanks Will for going through this.
> 
> Yes, from the logs I have, it looks like _resv callback is always called after
>  ->add_device. I will double check this with vfio bind case as well.
> 
> And I guess, this is what you are proposing to retrieve the smmu,
> 
> master = dev->iommu_fwspec->iommu_priv;
> smmu = master->smmu;
> 
> > > +	if (WARN_ON(!smmu))
> >
> > Again, how does this trigger?
> > >  		return;
> > >
> > > -	list_add_tail(&region->list, head);
> > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > +
> > > +		if (!is_of_node(smmu->dev->fwnode))
> > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > 
> > How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> > abstracted from the driver?
> 
> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
> this is a low priority for us at the moment. Is the suggestion is to have a common function
> outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
> is the best way here. 

Right, something like that. The driver shouldn't need to care whether or not
it's using ACPI or DT when setting these options.

Will

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

On Wed, Jul 05, 2017 at 09:48:53AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Will Deacon [mailto:will.deacon at arm.com]
> > Sent: Tuesday, July 04, 2017 6:38 PM
> > To: Shameerali Kolothum Thodi
> > Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
> > sudeep.holla at arm.com; robin.murphy at arm.com; hanjun.guo at linaro.org;
> > Gabriele Paoloni; John Garry; Linuxarm; linux-acpi at vger.kernel.org;
> > iommu at lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> > linux-arm-kernel at lists.infradead.org; devel at acpica.org
> > Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> > HiSilicon erratum 161010801
> > 
> > On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> > transactions.
> > >
> > > On these platforms GICv3 ITS translator is presented with the deviceID
> > > by extending the MSI payload data to 64 bits to include the deviceID.
> > > Hence, the PCIe controller on this platforms has to differentiate the
> > > MSI payload against other DMA payload and has to modify the MSI
> > payload.
> > > This basically makes it difficult for this platforms to have a SMMU
> > > translation for MSI.
> > >
> > > This patch implements a ACPI table based quirk to reserve the hw msi
> > > regions in the smmu-v3 driver which means these address regions will
> > > not be translated and will be excluded from iova allocations.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi@huawei.com>
> > > ---
> > >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> > >  1 file changed, 26 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> > v3.c
> > > index abe4b88..c9346f2 100644
> > > --- a/drivers/iommu/arm-smmu-v3.c
> > > +++ b/drivers/iommu/arm-smmu-v3.c
> > > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> > >  	u32				features;
> > >
> > >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> > >  	u32				options;
> > >
> > >  	struct arm_smmu_cmdq		cmdq;
> > > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> > device *dev,
> > >  				      struct list_head *head)
> > >  {
> > >  	struct iommu_resv_region *region;
> > > +	struct arm_smmu_device *smmu;
> > > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> > >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > > +	int resv = 0;
> > >
> > > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> > MSI_IOVA_LENGTH,
> > > -					 prot, IOMMU_RESV_SW_MSI);
> > > -	if (!region)
> > > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> > 
> > Does this callback actually get called without a prior ->add_device callback
> > for the master in question? If not, then we can already claw the structure
> > out via the iommu_priv field in the fwspec.
> 
> Thanks Will for going through this.
> 
> Yes, from the logs I have, it looks like _resv callback is always called after
>  ->add_device. I will double check this with vfio bind case as well.
> 
> And I guess, this is what you are proposing to retrieve the smmu,
> 
> master = dev->iommu_fwspec->iommu_priv;
> smmu = master->smmu;
> 
> > > +	if (WARN_ON(!smmu))
> >
> > Again, how does this trigger?
> > >  		return;
> > >
> > > -	list_add_tail(&region->list, head);
> > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > +
> > > +		if (!is_of_node(smmu->dev->fwnode))
> > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > 
> > How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> > abstracted from the driver?
> 
> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
> this is a low priority for us at the moment. Is the suggestion is to have a common function
> outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
> is the best way here. 

Right, something like that. The driver shouldn't need to care whether or not
it's using ACPI or DT when setting these options.

Will

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Will Deacon]

[-- Attachment #1: Type: text/plain, Size: 4277 bytes --]

On Wed, Jul 05, 2017 at 09:48:53AM +0000, Shameerali Kolothum Thodi wrote:
> 
> 
> > -----Original Message-----
> > From: Will Deacon [mailto:will.deacon(a)arm.com]
> > Sent: Tuesday, July 04, 2017 6:38 PM
> > To: Shameerali Kolothum Thodi
> > Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
> > sudeep.holla(a)arm.com; robin.murphy(a)arm.com; hanjun.guo(a)linaro.org;
> > Gabriele Paoloni; John Garry; Linuxarm; linux-acpi(a)vger.kernel.org;
> > iommu(a)lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> > linux-arm-kernel(a)lists.infradead.org; devel(a)acpica.org
> > Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> > HiSilicon erratum 161010801
> > 
> > On Fri, Jun 23, 2017 at 03:58:01PM +0100, shameer wrote:
> > > The HiSilicon erratum 161010801 describes the limitation of HiSilicon
> > > platforms Hip06/Hip07 to support the SMMU mappings for MSI
> > transactions.
> > >
> > > On these platforms GICv3 ITS translator is presented with the deviceID
> > > by extending the MSI payload data to 64 bits to include the deviceID.
> > > Hence, the PCIe controller on this platforms has to differentiate the
> > > MSI payload against other DMA payload and has to modify the MSI
> > payload.
> > > This basically makes it difficult for this platforms to have a SMMU
> > > translation for MSI.
> > >
> > > This patch implements a ACPI table based quirk to reserve the hw msi
> > > regions in the smmu-v3 driver which means these address regions will
> > > not be translated and will be excluded from iova allocations.
> > >
> > > Signed-off-by: shameer <shameerali.kolothum.thodi(a)huawei.com>
> > > ---
> > >  drivers/iommu/arm-smmu-v3.c | 30 ++++++++++++++++++++++++++----
> > >  1 file changed, 26 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-
> > v3.c
> > > index abe4b88..c9346f2 100644
> > > --- a/drivers/iommu/arm-smmu-v3.c
> > > +++ b/drivers/iommu/arm-smmu-v3.c
> > > @@ -597,6 +597,7 @@ struct arm_smmu_device {
> > >  	u32				features;
> > >
> > >  #define ARM_SMMU_OPT_SKIP_PREFETCH	(1 << 0)
> > > +#define ARM_SMMU_OPT_RESV_HW_MSI	(1 << 1)
> > >  	u32				options;
> > >
> > >  	struct arm_smmu_cmdq		cmdq;
> > > @@ -1904,14 +1905,34 @@ static void arm_smmu_get_resv_regions(struct
> > device *dev,
> > >  				      struct list_head *head)
> > >  {
> > >  	struct iommu_resv_region *region;
> > > +	struct arm_smmu_device *smmu;
> > > +	struct iommu_fwspec *fwspec = dev->iommu_fwspec;
> > >  	int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO;
> > > +	int resv = 0;
> > >
> > > -	region = iommu_alloc_resv_region(MSI_IOVA_BASE,
> > MSI_IOVA_LENGTH,
> > > -					 prot, IOMMU_RESV_SW_MSI);
> > > -	if (!region)
> > > +	smmu = arm_smmu_get_by_fwnode(fwspec->iommu_fwnode);
> > 
> > Does this callback actually get called without a prior ->add_device callback
> > for the master in question? If not, then we can already claw the structure
> > out via the iommu_priv field in the fwspec.
> 
> Thanks Will for going through this.
> 
> Yes, from the logs I have, it looks like _resv callback is always called after
>  ->add_device. I will double check this with vfio bind case as well.
> 
> And I guess, this is what you are proposing to retrieve the smmu,
> 
> master = dev->iommu_fwspec->iommu_priv;
> smmu = master->smmu;
> 
> > > +	if (WARN_ON(!smmu))
> >
> > Again, how does this trigger?
> > >  		return;
> > >
> > > -	list_add_tail(&region->list, head);
> > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > +
> > > +		if (!is_of_node(smmu->dev->fwnode))
> > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > 
> > How does this work when we're not using ACPI? Shouldn't of vs ACPI be
> > abstracted from the driver?
> 
> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT support for
> this is a low priority for us at the moment. Is the suggestion is to have a common function
> outside the smmu driver for _iommu_its_get_resv_regions() ? I am not sure what
> is the best way here. 

Right, something like that. The driver shouldn't need to care whether or not
it's using ACPI or DT when setting these options.

Will

RE: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon@arm.com]
> Sent: Friday, July 14, 2017 8:33 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
> sudeep.holla@arm.com; robin.murphy@arm.com; hanjun.guo@linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi@vger.kernel.org;
> iommu@lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel@lists.infradead.org; devel@acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
[...]
> > > > -	list_add_tail(&region->list, head);
> > > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > > +
> > > > +		if (!is_of_node(smmu->dev->fwnode))
> > > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > >
> > > How does this work when we're not using ACPI? Shouldn't of vs ACPI
> > > be abstracted from the driver?
> >
> > At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
> > support for this is a low priority for us at the moment. Is the
> > suggestion is to have a common function outside the smmu driver for
> > _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
> 
> Right, something like that. The driver shouldn't need to care whether or not
> it's using ACPI or DT when setting these options.

Below is what I have in mind for the common function for msi reserve.
But just wondering invoking iort_ functions from iommu code 
is acceptable or not . Could you please take a look and let me know.

Many thanks,
Shameer

--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -198,6 +199,24 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
+/**
+ * iommu_dma_get_msi_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions
+ * callback for HW MSI specific reservations. For now, this only
+ * covers ITS MSI region reservation using ACPI IORT helper function.
+ */
+int iommu_dma_get_msi_resv_regions(struct device *dev, struct list_head *list)
+{
+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode))
+		return iort_iommu_its_get_resv_regions(dev, list);
+
+	return -ENODEV;
+}
+EXPORT_SYMBOL(iommu_dma_get_msi_resv_regions);

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Will Deacon [mailto:will.deacon at arm.com]
> Sent: Friday, July 14, 2017 8:33 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
> sudeep.holla at arm.com; robin.murphy at arm.com; hanjun.guo at linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi at vger.kernel.org;
> iommu at lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel at lists.infradead.org; devel at acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
[...]
> > > > -	list_add_tail(&region->list, head);
> > > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > > +
> > > > +		if (!is_of_node(smmu->dev->fwnode))
> > > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > >
> > > How does this work when we're not using ACPI? Shouldn't of vs ACPI
> > > be abstracted from the driver?
> >
> > At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
> > support for this is a low priority for us at the moment. Is the
> > suggestion is to have a common function outside the smmu driver for
> > _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
> 
> Right, something like that. The driver shouldn't need to care whether or not
> it's using ACPI or DT when setting these options.

Below is what I have in mind for the common function for msi reserve.
But just wondering invoking iort_ functions from iommu code 
is acceptable or not . Could you please take a look and let me know.

Many thanks,
Shameer

--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -198,6 +199,24 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
+/**
+ * iommu_dma_get_msi_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions
+ * callback for HW MSI specific reservations. For now, this only
+ * covers ITS MSI region reservation using ACPI IORT helper function.
+ */
+int iommu_dma_get_msi_resv_regions(struct device *dev, struct list_head *list)
+{
+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode))
+		return iort_iommu_its_get_resv_regions(dev, list);
+
+	return -ENODEV;
+}
+EXPORT_SYMBOL(iommu_dma_get_msi_resv_regions);

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

[-- Attachment #1: Type: text/plain, Size: 2735 bytes --]



> -----Original Message-----
> From: Will Deacon [mailto:will.deacon(a)arm.com]
> Sent: Friday, July 14, 2017 8:33 PM
> To: Shameerali Kolothum Thodi
> Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
> sudeep.holla(a)arm.com; robin.murphy(a)arm.com; hanjun.guo(a)linaro.org;
> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi(a)vger.kernel.org;
> iommu(a)lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
> linux-arm-kernel(a)lists.infradead.org; devel(a)acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
[...]
> > > > -	list_add_tail(&region->list, head);
> > > > +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> > > > +
> > > > +		if (!is_of_node(smmu->dev->fwnode))
> > > > +			resv = iort_iommu_its_get_resv_regions(dev, head);
> > >
> > > How does this work when we're not using ACPI? Shouldn't of vs ACPI
> > > be abstracted from the driver?
> >
> > At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
> > support for this is a low priority for us at the moment. Is the
> > suggestion is to have a common function outside the smmu driver for
> > _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
> 
> Right, something like that. The driver shouldn't need to care whether or not
> it's using ACPI or DT when setting these options.

Below is what I have in mind for the common function for msi reserve.
But just wondering invoking iort_ functions from iommu code 
is acceptable or not . Could you please take a look and let me know.

Many thanks,
Shameer

--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */
 
+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -198,6 +199,24 @@ void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 }
 EXPORT_SYMBOL(iommu_dma_get_resv_regions);
 
+/**
+ * iommu_dma_get_msi_resv_regions - Reserved region driver helper
+ * @dev: Device from iommu_get_resv_regions()
+ * @list: Reserved region list from iommu_get_resv_regions()
+ *
+ * IOMMU drivers can use this to implement their .get_resv_regions
+ * callback for HW MSI specific reservations. For now, this only
+ * covers ITS MSI region reservation using ACPI IORT helper function.
+ */
+int iommu_dma_get_msi_resv_regions(struct device *dev, struct list_head *list)
+{
+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode))
+		return iort_iommu_its_get_resv_regions(dev, list);
+
+	return -ENODEV;
+}
+EXPORT_SYMBOL(iommu_dma_get_msi_resv_regions);

Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Robin Murphy]

On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Will Deacon [mailto:will.deacon@arm.com]
>> Sent: Friday, July 14, 2017 8:33 PM
>> To: Shameerali Kolothum Thodi
>> Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
>> sudeep.holla@arm.com; robin.murphy@arm.com; hanjun.guo@linaro.org;
>> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi@vger.kernel.org;
>> iommu@lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
>> linux-arm-kernel@lists.infradead.org; devel@acpica.org
>> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
>> HiSilicon erratum 161010801
>>
> [...]
>>>>> -	list_add_tail(&region->list, head);
>>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
>>>>> +
>>>>> +		if (!is_of_node(smmu->dev->fwnode))
>>>>> +			resv = iort_iommu_its_get_resv_regions(dev, head);
>>>>
>>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
>>>> be abstracted from the driver?
>>>
>>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
>>> support for this is a low priority for us at the moment. Is the
>>> suggestion is to have a common function outside the smmu driver for
>>> _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
>>
>> Right, something like that. The driver shouldn't need to care whether or not
>> it's using ACPI or DT when setting these options.
> 
> Below is what I have in mind for the common function for msi reserve.
> But just wondering invoking iort_ functions from iommu code 
> is acceptable or not . Could you please take a look and let me know.

At that point, it seems like we might as well just roll it into
iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
any DT equivalent to be described generically, rather than
SMMU-specific, so parsing that would fit into common code as well.

Then in the SMMU drivers we can skip creating the SW_MSI region if
iommu-dma gave us back any real ones (and remove the apparently
unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
doesn't seem to do much harm to have both.

Robin.

[1] This is what I hacked up locally on top of patch #1:
----->8-----
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 9d1cebe7f6cb..50292827da49 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */

+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device *dev,
struct list_head *list)
 	struct pci_host_bridge *bridge;
 	struct resource_entry *window;

+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
+		iort_iommu_its_get_resv_regions(dev, list) < 0)
+		return;
+
 	if (!dev_is_pci(dev))
 		return;

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Robin Murphy]

On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Will Deacon [mailto:will.deacon at arm.com]
>> Sent: Friday, July 14, 2017 8:33 PM
>> To: Shameerali Kolothum Thodi
>> Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
>> sudeep.holla at arm.com; robin.murphy at arm.com; hanjun.guo at linaro.org;
>> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi at vger.kernel.org;
>> iommu at lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
>> linux-arm-kernel at lists.infradead.org; devel at acpica.org
>> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
>> HiSilicon erratum 161010801
>>
> [...]
>>>>> -	list_add_tail(&region->list, head);
>>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
>>>>> +
>>>>> +		if (!is_of_node(smmu->dev->fwnode))
>>>>> +			resv = iort_iommu_its_get_resv_regions(dev, head);
>>>>
>>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
>>>> be abstracted from the driver?
>>>
>>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
>>> support for this is a low priority for us at the moment. Is the
>>> suggestion is to have a common function outside the smmu driver for
>>> _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
>>
>> Right, something like that. The driver shouldn't need to care whether or not
>> it's using ACPI or DT when setting these options.
> 
> Below is what I have in mind for the common function for msi reserve.
> But just wondering invoking iort_ functions from iommu code 
> is acceptable or not . Could you please take a look and let me know.

At that point, it seems like we might as well just roll it into
iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
any DT equivalent to be described generically, rather than
SMMU-specific, so parsing that would fit into common code as well.

Then in the SMMU drivers we can skip creating the SW_MSI region if
iommu-dma gave us back any real ones (and remove the apparently
unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
doesn't seem to do much harm to have both.

Robin.

[1] This is what I hacked up locally on top of patch #1:
----->8-----
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 9d1cebe7f6cb..50292827da49 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */

+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device *dev,
struct list_head *list)
 	struct pci_host_bridge *bridge;
 	struct resource_entry *window;

+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
+		iort_iommu_its_get_resv_regions(dev, list) < 0)
+		return;
+
 	if (!dev_is_pci(dev))
 		return;

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Robin Murphy]

[-- Attachment #1: Type: text/plain, Size: 2995 bytes --]

On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> 
> 
>> -----Original Message-----
>> From: Will Deacon [mailto:will.deacon(a)arm.com]
>> Sent: Friday, July 14, 2017 8:33 PM
>> To: Shameerali Kolothum Thodi
>> Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
>> sudeep.holla(a)arm.com; robin.murphy(a)arm.com; hanjun.guo(a)linaro.org;
>> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi(a)vger.kernel.org;
>> iommu(a)lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo);
>> linux-arm-kernel(a)lists.infradead.org; devel(a)acpica.org
>> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
>> HiSilicon erratum 161010801
>>
> [...]
>>>>> -	list_add_tail(&region->list, head);
>>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
>>>>> +
>>>>> +		if (!is_of_node(smmu->dev->fwnode))
>>>>> +			resv = iort_iommu_its_get_resv_regions(dev, head);
>>>>
>>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
>>>> be abstracted from the driver?
>>>
>>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and  DT
>>> support for this is a low priority for us at the moment. Is the
>>> suggestion is to have a common function outside the smmu driver for
>>> _iommu_its_get_resv_regions() ? I am not sure what is the best way here.
>>
>> Right, something like that. The driver shouldn't need to care whether or not
>> it's using ACPI or DT when setting these options.
> 
> Below is what I have in mind for the common function for msi reserve.
> But just wondering invoking iort_ functions from iommu code 
> is acceptable or not . Could you please take a look and let me know.

At that point, it seems like we might as well just roll it into
iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
any DT equivalent to be described generically, rather than
SMMU-specific, so parsing that would fit into common code as well.

Then in the SMMU drivers we can skip creating the SW_MSI region if
iommu-dma gave us back any real ones (and remove the apparently
unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
doesn't seem to do much harm to have both.

Robin.

[1] This is what I hacked up locally on top of patch #1:
----->8-----
diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
index 9d1cebe7f6cb..50292827da49 100644
--- a/drivers/iommu/dma-iommu.c
+++ b/drivers/iommu/dma-iommu.c
@@ -19,6 +19,7 @@
  * along with this program.  If not, see <http://www.gnu.org/licenses/>.
  */

+#include <linux/acpi_iort.h>
 #include <linux/device.h>
 #include <linux/dma-iommu.h>
 #include <linux/gfp.h>
@@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device *dev,
struct list_head *list)
 	struct pci_host_bridge *bridge;
 	struct resource_entry *window;

+	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
+		iort_iommu_its_get_resv_regions(dev, list) < 0)
+		return;
+
 	if (!dev_is_pci(dev))
 		return;

RE: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy@arm.com]
> Sent: Thursday, July 20, 2017 3:32 PM
> To: Shameerali Kolothum Thodi; Will Deacon
> Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
> sudeep.holla@arm.com; hanjun.guo@linaro.org; Gabriele Paoloni; John
> Garry; Linuxarm; linux-acpi@vger.kernel.org; iommu@lists.linux-
> foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo); linux-arm-
> kernel@lists.infradead.org; devel@acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> >
> >
> >> -----Original Message-----
> >> From: Will Deacon [mailto:will.deacon@arm.com]
> >> Sent: Friday, July 14, 2017 8:33 PM
> >> To: Shameerali Kolothum Thodi
> >> Cc: lorenzo.pieralisi@arm.com; marc.zyngier@arm.com;
> >> sudeep.holla@arm.com; robin.murphy@arm.com;
> hanjun.guo@linaro.org;
> >> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi@vger.kernel.org;
> >> iommu@lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun
> Guo);
> >> linux-arm-kernel@lists.infradead.org; devel@acpica.org
> >> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> >> HiSilicon erratum 161010801
> >>
> > [...]
> >>>>> -	list_add_tail(&region->list, head);
> >>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> >>>>> +
> >>>>> +		if (!is_of_node(smmu->dev->fwnode))
> >>>>> +			resv =
> iort_iommu_its_get_resv_regions(dev, head);
> >>>>
> >>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
> >>>> be abstracted from the driver?
> >>>
> >>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and
> DT
> >>> support for this is a low priority for us at the moment. Is the
> >>> suggestion is to have a common function outside the smmu driver for
> >>> _iommu_its_get_resv_regions() ? I am not sure what is the best way
> here.
> >>
> >> Right, something like that. The driver shouldn't need to care whether or
> not
> >> it's using ACPI or DT when setting these options.
> >
> > Below is what I have in mind for the common function for msi reserve.
> > But just wondering invoking iort_ functions from iommu code
> > is acceptable or not . Could you please take a look and let me know.
> 
> At that point, it seems like we might as well just roll it into
> iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
> any DT equivalent to be described generically, rather than
> SMMU-specific, so parsing that would fit into common code as well.
> 
> Then in the SMMU drivers we can skip creating the SW_MSI region if
> iommu-dma gave us back any real ones (and remove the apparently
> unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
> doesn't seem to do much harm to have both.

Ok, If I read that correctly, we don’t need any changes to SMMU driver for
now and this will be a generic change rather than a HiSi quirk.

So is it ok, if I just send the patch#1 rebased on 4.13-rc1?  

Thanks,
Shameer

> 
> [1] This is what I hacked up locally on top of patch #1:
> ----->8-----
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 9d1cebe7f6cb..50292827da49 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -19,6 +19,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
> 
> +#include <linux/acpi_iort.h>
>  #include <linux/device.h>
>  #include <linux/dma-iommu.h>
>  #include <linux/gfp.h>
> @@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device
> *dev,
> struct list_head *list)
>  	struct pci_host_bridge *bridge;
>  	struct resource_entry *window;
> 
> +	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
> +		iort_iommu_its_get_resv_regions(dev, list) < 0)
> +		return;
> +
>  	if (!dev_is_pci(dev))
>  		return;

[PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy at arm.com]
> Sent: Thursday, July 20, 2017 3:32 PM
> To: Shameerali Kolothum Thodi; Will Deacon
> Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
> sudeep.holla at arm.com; hanjun.guo at linaro.org; Gabriele Paoloni; John
> Garry; Linuxarm; linux-acpi at vger.kernel.org; iommu at lists.linux-
> foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo); linux-arm-
> kernel at lists.infradead.org; devel at acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> >
> >
> >> -----Original Message-----
> >> From: Will Deacon [mailto:will.deacon at arm.com]
> >> Sent: Friday, July 14, 2017 8:33 PM
> >> To: Shameerali Kolothum Thodi
> >> Cc: lorenzo.pieralisi at arm.com; marc.zyngier at arm.com;
> >> sudeep.holla at arm.com; robin.murphy at arm.com;
> hanjun.guo at linaro.org;
> >> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi at vger.kernel.org;
> >> iommu at lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun
> Guo);
> >> linux-arm-kernel at lists.infradead.org; devel at acpica.org
> >> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> >> HiSilicon erratum 161010801
> >>
> > [...]
> >>>>> -	list_add_tail(&region->list, head);
> >>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> >>>>> +
> >>>>> +		if (!is_of_node(smmu->dev->fwnode))
> >>>>> +			resv =
> iort_iommu_its_get_resv_regions(dev, head);
> >>>>
> >>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
> >>>> be abstracted from the driver?
> >>>
> >>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and
> DT
> >>> support for this is a low priority for us at the moment. Is the
> >>> suggestion is to have a common function outside the smmu driver for
> >>> _iommu_its_get_resv_regions() ? I am not sure what is the best way
> here.
> >>
> >> Right, something like that. The driver shouldn't need to care whether or
> not
> >> it's using ACPI or DT when setting these options.
> >
> > Below is what I have in mind for the common function for msi reserve.
> > But just wondering invoking iort_ functions from iommu code
> > is acceptable or not . Could you please take a look and let me know.
> 
> At that point, it seems like we might as well just roll it into
> iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
> any DT equivalent to be described generically, rather than
> SMMU-specific, so parsing that would fit into common code as well.
> 
> Then in the SMMU drivers we can skip creating the SW_MSI region if
> iommu-dma gave us back any real ones (and remove the apparently
> unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
> doesn't seem to do much harm to have both.

Ok, If I read that correctly, we don?t need any changes to SMMU driver for
now and this will be a generic change rather than a HiSi quirk.

So is it ok, if I just send the patch#1 rebased on 4.13-rc1?  

Thanks,
Shameer

> 
> [1] This is what I hacked up locally on top of patch #1:
> ----->8-----
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 9d1cebe7f6cb..50292827da49 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -19,6 +19,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
> 
> +#include <linux/acpi_iort.h>
>  #include <linux/device.h>
>  #include <linux/dma-iommu.h>
>  #include <linux/gfp.h>
> @@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device
> *dev,
> struct list_head *list)
>  	struct pci_host_bridge *bridge;
>  	struct resource_entry *window;
> 
> +	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
> +		iort_iommu_its_get_resv_regions(dev, list) < 0)
> +		return;
> +
>  	if (!dev_is_pci(dev))
>  		return;

Re: [Devel] [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based HiSilicon erratum 161010801

[Shameerali Kolothum Thodi]

[-- Attachment #1: Type: text/plain, Size: 3981 bytes --]



> -----Original Message-----
> From: Robin Murphy [mailto:robin.murphy(a)arm.com]
> Sent: Thursday, July 20, 2017 3:32 PM
> To: Shameerali Kolothum Thodi; Will Deacon
> Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
> sudeep.holla(a)arm.com; hanjun.guo(a)linaro.org; Gabriele Paoloni; John
> Garry; Linuxarm; linux-acpi(a)vger.kernel.org; iommu(a)lists.linux-
> foundation.org; Wangzhou (B); Guohanjun (Hanjun Guo); linux-arm-
> kernel(a)lists.infradead.org; devel(a)acpica.org
> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> HiSilicon erratum 161010801
> 
> On 19/07/17 11:48, Shameerali Kolothum Thodi wrote:
> >
> >
> >> -----Original Message-----
> >> From: Will Deacon [mailto:will.deacon(a)arm.com]
> >> Sent: Friday, July 14, 2017 8:33 PM
> >> To: Shameerali Kolothum Thodi
> >> Cc: lorenzo.pieralisi(a)arm.com; marc.zyngier(a)arm.com;
> >> sudeep.holla(a)arm.com; robin.murphy(a)arm.com;
> hanjun.guo(a)linaro.org;
> >> Gabriele Paoloni; John Garry; Linuxarm; linux-acpi(a)vger.kernel.org;
> >> iommu(a)lists.linux-foundation.org; Wangzhou (B); Guohanjun (Hanjun
> Guo);
> >> linux-arm-kernel(a)lists.infradead.org; devel(a)acpica.org
> >> Subject: Re: [PATCH v3 2/2] iommu/arm-smmu-v3:Enable ACPI based
> >> HiSilicon erratum 161010801
> >>
> > [...]
> >>>>> -	list_add_tail(&region->list, head);
> >>>>> +	if ((smmu->options & ARM_SMMU_OPT_RESV_HW_MSI)) {
> >>>>> +
> >>>>> +		if (!is_of_node(smmu->dev->fwnode))
> >>>>> +			resv =
> iort_iommu_its_get_resv_regions(dev, head);
> >>>>
> >>>> How does this work when we're not using ACPI? Shouldn't of vs ACPI
> >>>> be abstracted from the driver?
> >>>
> >>> At present ARM_SMMU_OPT_RESV_HW_MSI is only set for ACPI and
> DT
> >>> support for this is a low priority for us at the moment. Is the
> >>> suggestion is to have a common function outside the smmu driver for
> >>> _iommu_its_get_resv_regions() ? I am not sure what is the best way
> here.
> >>
> >> Right, something like that. The driver shouldn't need to care whether or
> not
> >> it's using ACPI or DT when setting these options.
> >
> > Below is what I have in mind for the common function for msi reserve.
> > But just wondering invoking iort_ functions from iommu code
> > is acceptable or not . Could you please take a look and let me know.
> 
> At that point, it seems like we might as well just roll it into
> iommu_dma_get_resv_regions() directly[1]. It probably makes sense for
> any DT equivalent to be described generically, rather than
> SMMU-specific, so parsing that would fit into common code as well.
> 
> Then in the SMMU drivers we can skip creating the SW_MSI region if
> iommu-dma gave us back any real ones (and remove the apparently
> unnecessary resv_msi check in VFIO). Or be lazy and just leave it, as it
> doesn't seem to do much harm to have both.

Ok, If I read that correctly, we don’t need any changes to SMMU driver for
now and this will be a generic change rather than a HiSi quirk.

So is it ok, if I just send the patch#1 rebased on 4.13-rc1?  

Thanks,
Shameer

> 
> [1] This is what I hacked up locally on top of patch #1:
> ----->8-----
> diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> index 9d1cebe7f6cb..50292827da49 100644
> --- a/drivers/iommu/dma-iommu.c
> +++ b/drivers/iommu/dma-iommu.c
> @@ -19,6 +19,7 @@
>   * along with this program.  If not, see <http://www.gnu.org/licenses/>.
>   */
> 
> +#include <linux/acpi_iort.h>
>  #include <linux/device.h>
>  #include <linux/dma-iommu.h>
>  #include <linux/gfp.h>
> @@ -174,6 +175,10 @@ void iommu_dma_get_resv_regions(struct device
> *dev,
> struct list_head *list)
>  	struct pci_host_bridge *bridge;
>  	struct resource_entry *window;
> 
> +	if (!is_of_node(dev->iommu_fwspec->iommu_fwnode) &&
> +		iort_iommu_its_get_resv_regions(dev, list) < 0)
> +		return;
> +
>  	if (!dev_is_pci(dev))
>  		return;