[RFC 0/9] IOMMU probe deferral support

[RFC 0/9] IOMMU probe deferral support

[Sricharan R]

This is mostly a repost of the probe deferral series from
Laurent Pinchart [1]. Added a check to fix boot with ACPI.
Adapted arm-smmu driver to work with deferred probing and added
a new api for the below reason. This is based on the generic iommu binding
series from Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>.

Now associating a group with an master has become mandatory and happens
when the master is added to the bus using BUS_ADD_DEVICE from iommu core.
But the iommu has to be ready before this using early iommu registration
and also master should have been added to the iommu using xlate. So, when
trying to get rid of the early registration and using late probing for iommu
devices to sort out the probing order, the newly added api is meant to be
called during the late probing to add the master to the iommu. add_iommu_group
could be modified to do the same though.

The one issue here is the DRIVER_BIND notifier from iommu core
might be called at a wrong point since the addition of a group to the device
happens after this.

Will be good to know the right direction to proceed on this fully.

[1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html

Laurent Pinchart (6):
  arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()
  of: dma: Move range size workaround to of_dma_get_range()
  of: dma: Make of_dma_deconfigure() public
  of: dma: Split of_configure_dma() into mask and ops configuration
  drivers: platform: Configure dma operations at probe time
  iommu: of: Handle IOMMU lookup failure with deferred probing or error

Sricharan R (3):
  drivers: iommu: Add a new add device api
  drivers: of: call iommu_bus_add_dev after iommu_configure_ops
  drivers: iommu: arm-smmu: Set iommu_ops in probe

 arch/arm/mm/dma-mapping.c |  9 ++++++
 drivers/base/platform.c   | 13 ++++++++
 drivers/iommu/arm-smmu.c  | 34 ++------------------
 drivers/iommu/iommu.c     | 12 +++++++
 drivers/iommu/of_iommu.c  | 16 ++++++++--
 drivers/of/address.c      | 20 ++++++++++--
 drivers/of/device.c       | 81 +++++++++++++++++++++++++++++++----------------
 drivers/of/platform.c     | 16 ++++------
 drivers/pci/probe.c       |  3 +-
 include/linux/iommu.h     |  1 +
 include/linux/of_device.h | 14 ++++++--
 11 files changed, 141 insertions(+), 78 deletions(-)

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 0/9] IOMMU probe deferral support

[Sricharan R]

This is mostly a repost of the probe deferral series from
Laurent Pinchart [1]. Added a check to fix boot with ACPI.
Adapted arm-smmu driver to work with deferred probing and added
a new api for the below reason. This is based on the generic iommu binding
series from Robin Murphy <robin.murphy@arm.com>.

Now associating a group with an master has become mandatory and happens
when the master is added to the bus using BUS_ADD_DEVICE from iommu core.
But the iommu has to be ready before this using early iommu registration
and also master should have been added to the iommu using xlate. So, when
trying to get rid of the early registration and using late probing for iommu
devices to sort out the probing order, the newly added api is meant to be
called during the late probing to add the master to the iommu. add_iommu_group
could be modified to do the same though.

The one issue here is the DRIVER_BIND notifier from iommu core
might be called at a wrong point since the addition of a group to the device
happens after this.

Will be good to know the right direction to proceed on this fully.

[1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html

Laurent Pinchart (6):
  arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()
  of: dma: Move range size workaround to of_dma_get_range()
  of: dma: Make of_dma_deconfigure() public
  of: dma: Split of_configure_dma() into mask and ops configuration
  drivers: platform: Configure dma operations at probe time
  iommu: of: Handle IOMMU lookup failure with deferred probing or error

Sricharan R (3):
  drivers: iommu: Add a new add device api
  drivers: of: call iommu_bus_add_dev after iommu_configure_ops
  drivers: iommu: arm-smmu: Set iommu_ops in probe

 arch/arm/mm/dma-mapping.c |  9 ++++++
 drivers/base/platform.c   | 13 ++++++++
 drivers/iommu/arm-smmu.c  | 34 ++------------------
 drivers/iommu/iommu.c     | 12 +++++++
 drivers/iommu/of_iommu.c  | 16 ++++++++--
 drivers/of/address.c      | 20 ++++++++++--
 drivers/of/device.c       | 81 +++++++++++++++++++++++++++++++----------------
 drivers/of/platform.c     | 16 ++++------
 drivers/pci/probe.c       |  3 +-
 include/linux/iommu.h     |  1 +
 include/linux/of_device.h | 14 ++++++--
 11 files changed, 141 insertions(+), 78 deletions(-)

-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 1/9] arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

The arch_setup_dma_ops() function is in charge of setting dma_ops with a
call to set_dma_ops(). set_dma_ops() is also called from

- highbank and mvebu bus notifiers
- dmabounce (to be replaced with swiotlb)
- arm_iommu_attach_device

(arm_iommu_attach_device is itself called from IOMMU and bus master
device drivers)

To allow the arch_setup_dma_ops() call to be moved from device add time
to device probe time we must ensure that dma_ops already setup by any of
the above callers will not be overriden.

Aftering replacing dmabounce with swiotlb, converting IOMMU drivers to
of_xlate and taking care of highbank and mvebu, the workaround should be
removed.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 arch/arm/mm/dma-mapping.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 0eca381..ccf60a2 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -2126,6 +2126,15 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
 	struct dma_map_ops *dma_ops;
 
 	dev->archdata.dma_coherent = coherent;
+
+	/*
+	 * Don't override the dma_ops if they have already been set. Ideally
+	 * this should be the only location where dma_ops are set, remove this
+	 * check when all other callers of set_dma_ops will have disappeared.
+	 */
+	if (dev->archdata.dma_ops)
+		return;
+
 	if (arm_setup_iommu_dma_ops(dev, dma_base, size, iommu))
 		dma_ops = arm_get_iommu_dma_map_ops(coherent);
 	else
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 1/9] arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

The arch_setup_dma_ops() function is in charge of setting dma_ops with a
call to set_dma_ops(). set_dma_ops() is also called from

- highbank and mvebu bus notifiers
- dmabounce (to be replaced with swiotlb)
- arm_iommu_attach_device

(arm_iommu_attach_device is itself called from IOMMU and bus master
device drivers)

To allow the arch_setup_dma_ops() call to be moved from device add time
to device probe time we must ensure that dma_ops already setup by any of
the above callers will not be overriden.

Aftering replacing dmabounce with swiotlb, converting IOMMU drivers to
of_xlate and taking care of highbank and mvebu, the workaround should be
removed.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 arch/arm/mm/dma-mapping.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
index 0eca381..ccf60a2 100644
--- a/arch/arm/mm/dma-mapping.c
+++ b/arch/arm/mm/dma-mapping.c
@@ -2126,6 +2126,15 @@ void arch_setup_dma_ops(struct device *dev, u64 dma_base, u64 size,
 	struct dma_map_ops *dma_ops;
 
 	dev->archdata.dma_coherent = coherent;
+
+	/*
+	 * Don't override the dma_ops if they have already been set. Ideally
+	 * this should be the only location where dma_ops are set, remove this
+	 * check when all other callers of set_dma_ops will have disappeared.
+	 */
+	if (dev->archdata.dma_ops)
+		return;
+
 	if (arm_setup_iommu_dma_ops(dev, dma_base, size, iommu))
 		dma_ops = arm_get_iommu_dma_map_ops(coherent);
 	else
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 2/9] of: dma: Move range size workaround to of_dma_get_range()

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

Invalid dma-ranges values should be worked around when retrieving the
DMA range in of_dma_get_range(), not by all callers of the function.
This isn't much of a problem now that we have a single caller, but that
situation will change when moving DMA configuration to device probe
time.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/of/address.c | 20 ++++++++++++++++++--
 drivers/of/device.c  | 15 ---------------
 2 files changed, 18 insertions(+), 17 deletions(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..c162a11 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -931,8 +931,8 @@ EXPORT_SYMBOL(of_io_request_and_map);
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
+ * this device in DT, or -EINVAL if the CPU address or size is invalid.
  */
 int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
 {
@@ -993,6 +993,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	*dma_addr = dmaaddr;
 
 	*size = of_read_number(ranges + naddr + pna, nsize);
+	/*
+	 * DT nodes sometimes incorrectly set the size as a mask. Work around
+	 * those incorrect DT by computing the size as mask + 1.
+	 */
+	if (*size & 1) {
+		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
+			__func__, *size, np->full_name);
+		*size = *size + 1;
+	}
+
+	if (!*size) {
+		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
+		       __func__, np->full_name);
+		ret = -EINVAL;
+		goto out;
+	}
 
 	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
 		 *dma_addr, *paddr, *size);
diff --git a/drivers/of/device.c b/drivers/of/device.c
index e5f47ce..fc7597c 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		size = dev->coherent_dma_mask + 1;
 	} else {
 		offset = PFN_DOWN(paddr - dma_addr);
-
-		/*
-		 * Add a work around to treat the size as mask + 1 in case
-		 * it is defined in DT as a mask.
-		 */
-		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
-			size = size + 1;
-		}
-
-		if (!size) {
-			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
-			return;
-		}
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 2/9] of: dma: Move range size workaround to of_dma_get_range()

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Invalid dma-ranges values should be worked around when retrieving the
DMA range in of_dma_get_range(), not by all callers of the function.
This isn't much of a problem now that we have a single caller, but that
situation will change when moving DMA configuration to device probe
time.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/of/address.c | 20 ++++++++++++++++++--
 drivers/of/device.c  | 15 ---------------
 2 files changed, 18 insertions(+), 17 deletions(-)

diff --git a/drivers/of/address.c b/drivers/of/address.c
index 91a469d..c162a11 100644
--- a/drivers/of/address.c
+++ b/drivers/of/address.c
@@ -931,8 +931,8 @@ EXPORT_SYMBOL(of_io_request_and_map);
  *	CPU addr (phys_addr_t)	: pna cells
  *	size			: nsize cells
  *
- * It returns -ENODEV if "dma-ranges" property was not found
- * for this device in DT.
+ * Return 0 on success, -ENODEV if the "dma-ranges" property was not found for
+ * this device in DT, or -EINVAL if the CPU address or size is invalid.
  */
 int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *size)
 {
@@ -993,6 +993,22 @@ int of_dma_get_range(struct device_node *np, u64 *dma_addr, u64 *paddr, u64 *siz
 	*dma_addr = dmaaddr;
 
 	*size = of_read_number(ranges + naddr + pna, nsize);
+	/*
+	 * DT nodes sometimes incorrectly set the size as a mask. Work around
+	 * those incorrect DT by computing the size as mask + 1.
+	 */
+	if (*size & 1) {
+		pr_warn("%s: size 0x%llx for dma-range in node(%s) set as mask\n",
+			__func__, *size, np->full_name);
+		*size = *size + 1;
+	}
+
+	if (!*size) {
+		pr_err("%s: invalid size zero for dma-range in node(%s)\n",
+		       __func__, np->full_name);
+		ret = -EINVAL;
+		goto out;
+	}
 
 	pr_debug("dma_addr(%llx) cpu_addr(%llx) size(%llx)\n",
 		 *dma_addr, *paddr, *size);
diff --git a/drivers/of/device.c b/drivers/of/device.c
index e5f47ce..fc7597c 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -110,21 +110,6 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		size = dev->coherent_dma_mask + 1;
 	} else {
 		offset = PFN_DOWN(paddr - dma_addr);
-
-		/*
-		 * Add a work around to treat the size as mask + 1 in case
-		 * it is defined in DT as a mask.
-		 */
-		if (size & 1) {
-			dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
-				 size);
-			size = size + 1;
-		}
-
-		if (!size) {
-			dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
-			return;
-		}
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 3/9] of: dma: Make of_dma_deconfigure() public

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

As part of moving DMA initializing to probe time the
of_dma_deconfigure() function will need to be called from different
source files. Make it public and move it to drivers/of/device.c where
the of_dma_configure() function is.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/of/device.c       | 12 ++++++++++++
 drivers/of/platform.c     |  5 -----
 include/linux/of_device.h |  3 +++
 3 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index fc7597c..c3e01fe 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -136,6 +136,18 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 }
 EXPORT_SYMBOL_GPL(of_dma_configure);
 
+/**
+ * of_dma_deconfigure - Clean up DMA configuration
+ * @dev:	Device for which to clean up DMA configuration
+ *
+ * Clean up all configuration performed by of_dma_configure_ops() and free all
+ * resources that have been allocated.
+ */
+void of_dma_deconfigure(struct device *dev)
+{
+	arch_teardown_dma_ops(dev);
+}
+
 int of_device_register(struct platform_device *pdev)
 {
 	device_initialize(&pdev->dev);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 8d103e4..329801b 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -151,11 +151,6 @@ struct platform_device *of_device_alloc(struct device_node *np,
 }
 EXPORT_SYMBOL(of_device_alloc);
 
-static void of_dma_deconfigure(struct device *dev)
-{
-	arch_teardown_dma_ops(dev);
-}
-
 /**
  * of_platform_device_create_pdata - Alloc, initialize and register an of_device
  * @np: pointer to node to create device for
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index cc7dd687..d20a31a 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -56,6 +56,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 
 void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
 static inline int of_driver_match_device(struct device *dev,
@@ -100,6 +101,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 static inline void of_dma_configure(struct device *dev, struct device_node *np)
 {}
+static inline void of_dma_deconfigure(struct device *dev)
+{}
 #endif /* CONFIG_OF */
 
 #endif /* _LINUX_OF_DEVICE_H */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 3/9] of: dma: Make of_dma_deconfigure() public

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

As part of moving DMA initializing to probe time the
of_dma_deconfigure() function will need to be called from different
source files. Make it public and move it to drivers/of/device.c where
the of_dma_configure() function is.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/of/device.c       | 12 ++++++++++++
 drivers/of/platform.c     |  5 -----
 include/linux/of_device.h |  3 +++
 3 files changed, 15 insertions(+), 5 deletions(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index fc7597c..c3e01fe 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -136,6 +136,18 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 }
 EXPORT_SYMBOL_GPL(of_dma_configure);
 
+/**
+ * of_dma_deconfigure - Clean up DMA configuration
+ * @dev:	Device for which to clean up DMA configuration
+ *
+ * Clean up all configuration performed by of_dma_configure_ops() and free all
+ * resources that have been allocated.
+ */
+void of_dma_deconfigure(struct device *dev)
+{
+	arch_teardown_dma_ops(dev);
+}
+
 int of_device_register(struct platform_device *pdev)
 {
 	device_initialize(&pdev->dev);
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 8d103e4..329801b 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -151,11 +151,6 @@ struct platform_device *of_device_alloc(struct device_node *np,
 }
 EXPORT_SYMBOL(of_device_alloc);
 
-static void of_dma_deconfigure(struct device *dev)
-{
-	arch_teardown_dma_ops(dev);
-}
-
 /**
  * of_platform_device_create_pdata - Alloc, initialize and register an of_device
  * @np: pointer to node to create device for
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index cc7dd687..d20a31a 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -56,6 +56,7 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 
 void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
 static inline int of_driver_match_device(struct device *dev,
@@ -100,6 +101,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 }
 static inline void of_dma_configure(struct device *dev, struct device_node *np)
 {}
+static inline void of_dma_deconfigure(struct device *dev)
+{}
 #endif /* CONFIG_OF */
 
 #endif /* _LINUX_OF_DEVICE_H */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 4/9] of: dma: Split of_configure_dma() into mask and ops configuration

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

The of_configure_dma() function configures both the DMA masks and ops.
Moving DMA ops configuration to probe time would thus also delay
configuration of the DMA masks, which might not be safe. To avoid issues
split the configuration in two to allow keeping masks configuration at
device add time and move ops configuration to device probe time.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/of/device.c       | 48 ++++++++++++++++++++++++++++++++++-------------
 drivers/of/platform.c     |  6 ++++--
 drivers/pci/probe.c       |  3 ++-
 include/linux/of_device.h | 11 +++++++++--
 4 files changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index c3e01fe..494cdbe 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -71,7 +71,7 @@ int of_device_add(struct platform_device *ofdev)
 }
 
 /**
- * of_dma_configure - Setup DMA configuration
+ * of_dma_configure - Setup DMA masks and offset
  * @dev:	Device to apply DMA configuration
  * @np:		Pointer to OF node having DMA configuration
  *
@@ -82,13 +82,11 @@ int of_device_add(struct platform_device *ofdev)
  * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
  * to fix up DMA configuration.
  */
-void of_dma_configure(struct device *dev, struct device_node *np)
+void of_dma_configure_masks(struct device *dev, struct device_node *np)
 {
-	u64 dma_addr, paddr, size;
-	int ret;
-	bool coherent;
+	u64 dma_addr, paddr, size, range_mask;
 	unsigned long offset;
-	struct iommu_ops *iommu;
+	int ret;
 
 	/*
 	 * Set default coherent_dma_mask to 32 bit.  Drivers are expected to
@@ -106,9 +104,10 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 
 	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
 	if (ret < 0) {
-		dma_addr = offset = 0;
-		size = dev->coherent_dma_mask + 1;
+		range_mask = dev->coherent_dma_mask + 1;
+		offset = 0;
 	} else {
+		range_mask = DMA_BIT_MASK(ilog2(dma_addr + size));
 		offset = PFN_DOWN(paddr - dma_addr);
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
@@ -119,10 +118,31 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 	 * Limit coherent and dma mask based on size and default mask
 	 * set by the driver.
 	 */
-	dev->coherent_dma_mask = min(dev->coherent_dma_mask,
-				     DMA_BIT_MASK(ilog2(dma_addr + size)));
-	*dev->dma_mask = min((*dev->dma_mask),
-			     DMA_BIT_MASK(ilog2(dma_addr + size)));
+	dev->coherent_dma_mask = min(dev->coherent_dma_mask, range_mask);
+	*dev->dma_mask = min((*dev->dma_mask), range_mask);
+}
+EXPORT_SYMBOL_GPL(of_dma_configure_masks);
+
+/**
+ * of_dma_configure_ops - Setup DMA operations
+ * @dev:	Device to apply DMA configuration
+ * @np:		Pointer to OF node having DMA configuration
+ *
+ * Try to get devices's DMA configuration from DT and update it
+ * accordingly.
+ */
+int of_dma_configure_ops(struct device *dev, struct device_node *np)
+{
+	u64 dma_addr, paddr, size;
+	struct iommu_ops *iommu;
+	bool coherent;
+	int ret;
+
+	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	if (ret < 0) {
+		dma_addr = 0;
+		size = dev->coherent_dma_mask + 1;
+	}
 
 	coherent = of_dma_is_coherent(np);
 	dev_dbg(dev, "device is%sdma coherent\n",
@@ -133,8 +153,10 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		iommu ? " " : " not ");
 
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+
+	return 0;
 }
-EXPORT_SYMBOL_GPL(of_dma_configure);
+EXPORT_SYMBOL_GPL(of_dma_configure_ops);
 
 /**
  * of_dma_deconfigure - Clean up DMA configuration
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 329801b..17ee8d5 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -179,7 +179,8 @@ static struct platform_device *of_platform_device_create_pdata(
 
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
-	of_dma_configure(&dev->dev, dev->dev.of_node);
+	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
+	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 	of_msi_configure(&dev->dev, dev->dev.of_node);
 
 	if (of_device_add(dev) != 0) {
@@ -243,7 +244,8 @@ static struct amba_device *of_amba_device_create(struct device_node *node,
 		dev_set_name(&dev->dev, "%s", bus_id);
 	else
 		of_device_make_bus_id(&dev->dev);
-	of_dma_configure(&dev->dev, dev->dev.of_node);
+	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
+	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 
 	/* Allow the HW Peripheral ID to be overridden */
 	prop = of_get_property(node, "arm,primecell-periphid", NULL);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6d7ab9b..90bde90 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1682,7 +1682,8 @@ static void pci_dma_configure(struct pci_dev *dev)
 
 	if (IS_ENABLED(CONFIG_OF) &&
 		bridge->parent && bridge->parent->of_node) {
-			of_dma_configure(&dev->dev, bridge->parent->of_node);
+		of_dma_configure_masks(&dev->dev, bridge->parent->of_node);
+		of_dma_configure_ops(&dev->dev, bridge->parent->of_node);
 	} else if (has_acpi_companion(bridge)) {
 		struct acpi_device *adev = to_acpi_device_node(bridge->fwnode);
 		enum dev_dma_attr attr = acpi_get_dma_attr(adev);
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index d20a31a..4b8cb7d 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -55,7 +55,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 	return of_node_get(cpu_dev->of_node);
 }
 
-void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_configure_masks(struct device *dev, struct device_node *np);
+int of_dma_configure_ops(struct device *dev, struct device_node *np);
 void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
@@ -99,8 +100,14 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 {
 	return NULL;
 }
-static inline void of_dma_configure(struct device *dev, struct device_node *np)
+static inline void of_dma_configure_masks(struct device *dev,
+					  struct device_node *np)
 {}
+static inline int of_dma_configure_ops(struct device *dev,
+				       struct device_node *np)
+{
+	return 0;
+}
 static inline void of_dma_deconfigure(struct device *dev)
 {}
 #endif /* CONFIG_OF */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 4/9] of: dma: Split of_configure_dma() into mask and ops configuration

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

The of_configure_dma() function configures both the DMA masks and ops.
Moving DMA ops configuration to probe time would thus also delay
configuration of the DMA masks, which might not be safe. To avoid issues
split the configuration in two to allow keeping masks configuration at
device add time and move ops configuration to device probe time.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/of/device.c       | 48 ++++++++++++++++++++++++++++++++++-------------
 drivers/of/platform.c     |  6 ++++--
 drivers/pci/probe.c       |  3 ++-
 include/linux/of_device.h | 11 +++++++++--
 4 files changed, 50 insertions(+), 18 deletions(-)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index c3e01fe..494cdbe 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -71,7 +71,7 @@ int of_device_add(struct platform_device *ofdev)
 }
 
 /**
- * of_dma_configure - Setup DMA configuration
+ * of_dma_configure - Setup DMA masks and offset
  * @dev:	Device to apply DMA configuration
  * @np:		Pointer to OF node having DMA configuration
  *
@@ -82,13 +82,11 @@ int of_device_add(struct platform_device *ofdev)
  * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
  * to fix up DMA configuration.
  */
-void of_dma_configure(struct device *dev, struct device_node *np)
+void of_dma_configure_masks(struct device *dev, struct device_node *np)
 {
-	u64 dma_addr, paddr, size;
-	int ret;
-	bool coherent;
+	u64 dma_addr, paddr, size, range_mask;
 	unsigned long offset;
-	struct iommu_ops *iommu;
+	int ret;
 
 	/*
 	 * Set default coherent_dma_mask to 32 bit.  Drivers are expected to
@@ -106,9 +104,10 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 
 	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
 	if (ret < 0) {
-		dma_addr = offset = 0;
-		size = dev->coherent_dma_mask + 1;
+		range_mask = dev->coherent_dma_mask + 1;
+		offset = 0;
 	} else {
+		range_mask = DMA_BIT_MASK(ilog2(dma_addr + size));
 		offset = PFN_DOWN(paddr - dma_addr);
 		dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
 	}
@@ -119,10 +118,31 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 	 * Limit coherent and dma mask based on size and default mask
 	 * set by the driver.
 	 */
-	dev->coherent_dma_mask = min(dev->coherent_dma_mask,
-				     DMA_BIT_MASK(ilog2(dma_addr + size)));
-	*dev->dma_mask = min((*dev->dma_mask),
-			     DMA_BIT_MASK(ilog2(dma_addr + size)));
+	dev->coherent_dma_mask = min(dev->coherent_dma_mask, range_mask);
+	*dev->dma_mask = min((*dev->dma_mask), range_mask);
+}
+EXPORT_SYMBOL_GPL(of_dma_configure_masks);
+
+/**
+ * of_dma_configure_ops - Setup DMA operations
+ * @dev:	Device to apply DMA configuration
+ * @np:		Pointer to OF node having DMA configuration
+ *
+ * Try to get devices's DMA configuration from DT and update it
+ * accordingly.
+ */
+int of_dma_configure_ops(struct device *dev, struct device_node *np)
+{
+	u64 dma_addr, paddr, size;
+	struct iommu_ops *iommu;
+	bool coherent;
+	int ret;
+
+	ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+	if (ret < 0) {
+		dma_addr = 0;
+		size = dev->coherent_dma_mask + 1;
+	}
 
 	coherent = of_dma_is_coherent(np);
 	dev_dbg(dev, "device is%sdma coherent\n",
@@ -133,8 +153,10 @@ void of_dma_configure(struct device *dev, struct device_node *np)
 		iommu ? " " : " not ");
 
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+
+	return 0;
 }
-EXPORT_SYMBOL_GPL(of_dma_configure);
+EXPORT_SYMBOL_GPL(of_dma_configure_ops);
 
 /**
  * of_dma_deconfigure - Clean up DMA configuration
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 329801b..17ee8d5 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -179,7 +179,8 @@ static struct platform_device *of_platform_device_create_pdata(
 
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
-	of_dma_configure(&dev->dev, dev->dev.of_node);
+	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
+	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 	of_msi_configure(&dev->dev, dev->dev.of_node);
 
 	if (of_device_add(dev) != 0) {
@@ -243,7 +244,8 @@ static struct amba_device *of_amba_device_create(struct device_node *node,
 		dev_set_name(&dev->dev, "%s", bus_id);
 	else
 		of_device_make_bus_id(&dev->dev);
-	of_dma_configure(&dev->dev, dev->dev.of_node);
+	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
+	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 
 	/* Allow the HW Peripheral ID to be overridden */
 	prop = of_get_property(node, "arm,primecell-periphid", NULL);
diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index 6d7ab9b..90bde90 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1682,7 +1682,8 @@ static void pci_dma_configure(struct pci_dev *dev)
 
 	if (IS_ENABLED(CONFIG_OF) &&
 		bridge->parent && bridge->parent->of_node) {
-			of_dma_configure(&dev->dev, bridge->parent->of_node);
+		of_dma_configure_masks(&dev->dev, bridge->parent->of_node);
+		of_dma_configure_ops(&dev->dev, bridge->parent->of_node);
 	} else if (has_acpi_companion(bridge)) {
 		struct acpi_device *adev = to_acpi_device_node(bridge->fwnode);
 		enum dev_dma_attr attr = acpi_get_dma_attr(adev);
diff --git a/include/linux/of_device.h b/include/linux/of_device.h
index d20a31a..4b8cb7d 100644
--- a/include/linux/of_device.h
+++ b/include/linux/of_device.h
@@ -55,7 +55,8 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 	return of_node_get(cpu_dev->of_node);
 }
 
-void of_dma_configure(struct device *dev, struct device_node *np);
+void of_dma_configure_masks(struct device *dev, struct device_node *np);
+int of_dma_configure_ops(struct device *dev, struct device_node *np);
 void of_dma_deconfigure(struct device *dev);
 #else /* CONFIG_OF */
 
@@ -99,8 +100,14 @@ static inline struct device_node *of_cpu_device_node_get(int cpu)
 {
 	return NULL;
 }
-static inline void of_dma_configure(struct device *dev, struct device_node *np)
+static inline void of_dma_configure_masks(struct device *dev,
+					  struct device_node *np)
 {}
+static inline int of_dma_configure_ops(struct device *dev,
+				       struct device_node *np)
+{
+	return 0;
+}
 static inline void of_dma_deconfigure(struct device *dev)
 {}
 #endif /* CONFIG_OF */
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 5/9] drivers: platform: Configure dma operations at probe time

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

Configuring DMA ops at probe time will allow deferring device probe when
the IOMMU isn't available yet.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/base/platform.c | 13 +++++++++++++
 drivers/of/platform.c   |  7 +++----
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/drivers/base/platform.c b/drivers/base/platform.c
index f437afa..17e64a4 100644
--- a/drivers/base/platform.c
+++ b/drivers/base/platform.c
@@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
 	if (ret < 0)
 		return ret;
 
+	if (of_have_populated_dt()) {
+		ret = of_dma_configure_ops(_dev, _dev->of_node);
+		if (ret < 0)
+			goto done;
+	}
+
 	ret = dev_pm_domain_attach(_dev, true);
 	if (ret != -EPROBE_DEFER) {
 		if (drv->probe) {
@@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
 		}
 	}
 
+	if (of_have_populated_dt()) {
+		if (ret)
+			of_dma_deconfigure(_dev);
+	}
+done:
 	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
 		dev_warn(_dev, "probe deferral not supported\n");
 		ret = -ENXIO;
@@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
 	if (drv->remove)
 		ret = drv->remove(dev);
 	dev_pm_domain_detach(_dev, true);
+	if (of_have_populated_dt())
+		of_dma_deconfigure(_dev);
 
 	return ret;
 }
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 17ee8d5..12bbc8e1 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
 	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
-	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 	of_msi_configure(&dev->dev, dev->dev.of_node);
 
 	if (of_device_add(dev) != 0) {
-		of_dma_deconfigure(&dev->dev);
 		platform_device_put(dev);
 		goto err_clear_flag;
 	}
@@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
 	if (dev->bus == &platform_bus_type)
 		platform_device_unregister(to_platform_device(dev));
 #ifdef CONFIG_ARM_AMBA
-	else if (dev->bus == &amba_bustype)
+	else if (dev->bus == &amba_bustype) {
 		amba_device_unregister(to_amba_device(dev));
+		of_dma_deconfigure(dev);
+	}
 #endif
 
-	of_dma_deconfigure(dev);
 	of_node_clear_flag(dev->of_node, OF_POPULATED);
 	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
 	return 0;
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 5/9] drivers: platform: Configure dma operations at probe time

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Configuring DMA ops at probe time will allow deferring device probe when
the IOMMU isn't available yet.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/base/platform.c | 13 +++++++++++++
 drivers/of/platform.c   |  7 +++----
 2 files changed, 16 insertions(+), 4 deletions(-)

diff --git a/drivers/base/platform.c b/drivers/base/platform.c
index f437afa..17e64a4 100644
--- a/drivers/base/platform.c
+++ b/drivers/base/platform.c
@@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
 	if (ret < 0)
 		return ret;
 
+	if (of_have_populated_dt()) {
+		ret = of_dma_configure_ops(_dev, _dev->of_node);
+		if (ret < 0)
+			goto done;
+	}
+
 	ret = dev_pm_domain_attach(_dev, true);
 	if (ret != -EPROBE_DEFER) {
 		if (drv->probe) {
@@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
 		}
 	}
 
+	if (of_have_populated_dt()) {
+		if (ret)
+			of_dma_deconfigure(_dev);
+	}
+done:
 	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
 		dev_warn(_dev, "probe deferral not supported\n");
 		ret = -ENXIO;
@@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
 	if (drv->remove)
 		ret = drv->remove(dev);
 	dev_pm_domain_detach(_dev, true);
+	if (of_have_populated_dt())
+		of_dma_deconfigure(_dev);
 
 	return ret;
 }
diff --git a/drivers/of/platform.c b/drivers/of/platform.c
index 17ee8d5..12bbc8e1 100644
--- a/drivers/of/platform.c
+++ b/drivers/of/platform.c
@@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
 	dev->dev.bus = &platform_bus_type;
 	dev->dev.platform_data = platform_data;
 	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
-	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
 	of_msi_configure(&dev->dev, dev->dev.of_node);
 
 	if (of_device_add(dev) != 0) {
-		of_dma_deconfigure(&dev->dev);
 		platform_device_put(dev);
 		goto err_clear_flag;
 	}
@@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
 	if (dev->bus == &platform_bus_type)
 		platform_device_unregister(to_platform_device(dev));
 #ifdef CONFIG_ARM_AMBA
-	else if (dev->bus == &amba_bustype)
+	else if (dev->bus == &amba_bustype) {
 		amba_device_unregister(to_amba_device(dev));
+		of_dma_deconfigure(dev);
+	}
 #endif
 
-	of_dma_deconfigure(dev);
 	of_node_clear_flag(dev->of_node, OF_POPULATED);
 	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
 	return 0;
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 6/9] iommu: of: Handle IOMMU lookup failure with deferred probing or error

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>

Failures to look up an IOMMU when parsing the DT iommus property need to
be handled separately from the .of_xlate() failures to support deferred
probing.

The lack of a registered IOMMU can be caused by the lack of a driver for
the IOMMU, the IOMMU device probe not having been performed yet, having
been deferred, or having failed.

The first case occurs when the device tree describes the bus master and
IOMMU topology correctly but no device driver exists for the IOMMU yet
or the device driver has not been compiled in. Return NULL, the caller
will configure the device without an IOMMU.

The second and third cases are handled by deferring the probe of the bus
master device which will eventually get reprobed after the IOMMU.

The last case is currently handled by deferring the probe of the bus
master device as well. A mechanism to either configure the bus master
device without an IOMMU or to fail the bus master device probe depending
on whether the IOMMU is optional or mandatory would be a good
enhancement.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
---
 drivers/iommu/of_iommu.c | 16 +++++++++++++---
 drivers/of/device.c      |  2 ++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 910826c..f722f55 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -189,8 +189,18 @@ struct iommu_ops *of_iommu_configure(struct device *dev,
 		np = iommu_spec.np;
 		ops = of_iommu_get_ops(np);
 
-		if (!ops || !ops->of_xlate || ops->of_xlate(dev, &iommu_spec))
+		if (!ops) {
+			const struct of_device_id *oid;
+
+			oid = of_match_node(&__iommu_of_table, np);
+			ops = oid ? ERR_PTR(-EPROBE_DEFER) : NULL;
 			goto err_put_node;
+		}
+
+		if (!ops->of_xlate || ops->of_xlate(dev, &iommu_spec)) {
+			ops = NULL;
+			goto err_put_node;
+		}
 
 		of_node_put(np);
 		idx++;
@@ -200,7 +210,7 @@ struct iommu_ops *of_iommu_configure(struct device *dev,
 
 err_put_node:
 	of_node_put(np);
-	return NULL;
+	return ops;
 }
 
 void __init of_iommu_init(void)
@@ -211,7 +221,7 @@ void __init of_iommu_init(void)
 	for_each_matching_node_and_match(np, matches, &match) {
 		const of_iommu_init_fn init_fn = match->data;
 
-		if (init_fn(np))
+		if (init_fn && init_fn(np))
 			pr_err("Failed to initialise IOMMU %s\n",
 				of_node_full_name(np));
 	}
diff --git a/drivers/of/device.c b/drivers/of/device.c
index 494cdbe..57a5f2d 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -149,6 +149,8 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np);
+	if (IS_ERR(iommu))
+		return PTR_ERR(iommu);
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 6/9] iommu: of: Handle IOMMU lookup failure with deferred probing or error

[Sricharan R]

From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>

Failures to look up an IOMMU when parsing the DT iommus property need to
be handled separately from the .of_xlate() failures to support deferred
probing.

The lack of a registered IOMMU can be caused by the lack of a driver for
the IOMMU, the IOMMU device probe not having been performed yet, having
been deferred, or having failed.

The first case occurs when the device tree describes the bus master and
IOMMU topology correctly but no device driver exists for the IOMMU yet
or the device driver has not been compiled in. Return NULL, the caller
will configure the device without an IOMMU.

The second and third cases are handled by deferring the probe of the bus
master device which will eventually get reprobed after the IOMMU.

The last case is currently handled by deferring the probe of the bus
master device as well. A mechanism to either configure the bus master
device without an IOMMU or to fail the bus master device probe depending
on whether the IOMMU is optional or mandatory would be a good
enhancement.

Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
---
 drivers/iommu/of_iommu.c | 16 +++++++++++++---
 drivers/of/device.c      |  2 ++
 2 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c
index 910826c..f722f55 100644
--- a/drivers/iommu/of_iommu.c
+++ b/drivers/iommu/of_iommu.c
@@ -189,8 +189,18 @@ struct iommu_ops *of_iommu_configure(struct device *dev,
 		np = iommu_spec.np;
 		ops = of_iommu_get_ops(np);
 
-		if (!ops || !ops->of_xlate || ops->of_xlate(dev, &iommu_spec))
+		if (!ops) {
+			const struct of_device_id *oid;
+
+			oid = of_match_node(&__iommu_of_table, np);
+			ops = oid ? ERR_PTR(-EPROBE_DEFER) : NULL;
 			goto err_put_node;
+		}
+
+		if (!ops->of_xlate || ops->of_xlate(dev, &iommu_spec)) {
+			ops = NULL;
+			goto err_put_node;
+		}
 
 		of_node_put(np);
 		idx++;
@@ -200,7 +210,7 @@ struct iommu_ops *of_iommu_configure(struct device *dev,
 
 err_put_node:
 	of_node_put(np);
-	return NULL;
+	return ops;
 }
 
 void __init of_iommu_init(void)
@@ -211,7 +221,7 @@ void __init of_iommu_init(void)
 	for_each_matching_node_and_match(np, matches, &match) {
 		const of_iommu_init_fn init_fn = match->data;
 
-		if (init_fn(np))
+		if (init_fn && init_fn(np))
 			pr_err("Failed to initialise IOMMU %s\n",
 				of_node_full_name(np));
 	}
diff --git a/drivers/of/device.c b/drivers/of/device.c
index 494cdbe..57a5f2d 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -149,6 +149,8 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
 		coherent ? " " : " not ");
 
 	iommu = of_iommu_configure(dev, np);
+	if (IS_ERR(iommu))
+		return PTR_ERR(iommu);
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 7/9] drivers: iommu: Add a new add device api

[Sricharan R]

Associating a master with an group happens when the master is added to the bus,
notified with BUS_ADD_DEVICE. But the iommu has to be ready before this
using early iommu registration and also master should have been added to the
iommu using xlate. So, when trying to get rid of the early registration and
using late probing for iommu devices to sort out the probing order, the newly
added api is meant to be called during the late probing to add the master
to iommu.

Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/iommu/iommu.c | 12 ++++++++++++
 include/linux/iommu.h |  1 +
 2 files changed, 13 insertions(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bfd4f7c..a0dd01f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -900,6 +900,18 @@ static int remove_iommu_group(struct device *dev, void *data)
 	return 0;
 }
 
+int iommu_bus_add_dev(struct device *dev)
+{
+	const struct iommu_ops *ops = dev->bus->iommu_ops;
+	int ret = -ENODEV;
+
+	if (ops->add_device)
+		ret = ops->add_device(dev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_bus_add_dev);
+
 static int iommu_bus_notifier(struct notifier_block *nb,
 			      unsigned long action, void *data)
 {
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index a5c539f..018cfc8 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -271,6 +271,7 @@ struct device *iommu_device_create(struct device *parent, void *drvdata,
 void iommu_device_destroy(struct device *dev);
 int iommu_device_link(struct device *dev, struct device *link);
 void iommu_device_unlink(struct device *dev, struct device *link);
+int iommu_bus_add_dev(struct device *dev);
 
 /* Window handling function prototypes */
 extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 7/9] drivers: iommu: Add a new add device api

[Sricharan R]

Associating a master with an group happens when the master is added to the bus,
notified with BUS_ADD_DEVICE. But the iommu has to be ready before this
using early iommu registration and also master should have been added to the
iommu using xlate. So, when trying to get rid of the early registration and
using late probing for iommu devices to sort out the probing order, the newly
added api is meant to be called during the late probing to add the master
to iommu.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
---
 drivers/iommu/iommu.c | 12 ++++++++++++
 include/linux/iommu.h |  1 +
 2 files changed, 13 insertions(+)

diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bfd4f7c..a0dd01f 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -900,6 +900,18 @@ static int remove_iommu_group(struct device *dev, void *data)
 	return 0;
 }
 
+int iommu_bus_add_dev(struct device *dev)
+{
+	const struct iommu_ops *ops = dev->bus->iommu_ops;
+	int ret = -ENODEV;
+
+	if (ops->add_device)
+		ret = ops->add_device(dev);
+
+	return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_bus_add_dev);
+
 static int iommu_bus_notifier(struct notifier_block *nb,
 			      unsigned long action, void *data)
 {
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index a5c539f..018cfc8 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -271,6 +271,7 @@ struct device *iommu_device_create(struct device *parent, void *drvdata,
 void iommu_device_destroy(struct device *dev);
 int iommu_device_link(struct device *dev, struct device *link);
 void iommu_device_unlink(struct device *dev, struct device *link);
+int iommu_bus_add_dev(struct device *dev);
 
 /* Window handling function prototypes */
 extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan R]

Now that the device's iommu ops are configured at probe time,
the device has to be added to the iommu late.

Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/of/device.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index 57a5f2d..722115c 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -6,6 +6,7 @@
 #include <linux/of_iommu.h>
 #include <linux/dma-mapping.h>
 #include <linux/init.h>
+#include <linux/iommu.h>
 #include <linux/module.h>
 #include <linux/mod_devicetable.h>
 #include <linux/slab.h>
@@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
+	if (iommu)
+		iommu_bus_add_dev(dev);
+
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
 
 	return 0;
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan R]

Now that the device's iommu ops are configured at probe time,
the device has to be added to the iommu late.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
---
 drivers/of/device.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/of/device.c b/drivers/of/device.c
index 57a5f2d..722115c 100644
--- a/drivers/of/device.c
+++ b/drivers/of/device.c
@@ -6,6 +6,7 @@
 #include <linux/of_iommu.h>
 #include <linux/dma-mapping.h>
 #include <linux/init.h>
+#include <linux/iommu.h>
 #include <linux/module.h>
 #include <linux/mod_devicetable.h>
 #include <linux/slab.h>
@@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
 	dev_dbg(dev, "device is%sbehind an iommu\n",
 		iommu ? " " : " not ");
 
+	if (iommu)
+		iommu_bus_add_dev(dev);
+
 	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
 
 	return 0;
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 9/9] drivers: iommu: arm-smmu: Set iommu_ops in probe

[Sricharan R]

iommu_ops should be set after the iommu is probed.
This ensures that the iommu is really ready when master's
iommu ops are set during their probe or else deferred.

Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
---
 drivers/iommu/arm-smmu.c | 34 ++--------------------------------
 1 file changed, 2 insertions(+), 32 deletions(-)

diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 2d65de4..f103d35 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1919,6 +1919,7 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 	if (of_property_read_bool(dev->of_node, "mmu-masters"))
 		arm_smmu_probe_mmu_masters(smmu);
 	arm_smmu_device_reset(smmu);
+	of_iommu_set_ops(dev->of_node, &arm_smmu_ops);
 	return 0;
 
 out_free_irqs:
@@ -1957,24 +1958,13 @@ static struct platform_driver arm_smmu_driver = {
 	.remove	= arm_smmu_device_remove,
 };
 
-static int __init arm_smmu_init(void)
+static int __init arm_smmu_of_init(struct device_node *np)
 {
-	struct device_node *np;
 	static bool done;
 	int ret;
 
 	if (done)
 		return 0;
-	/*
-	 * Play nice with systems that don't have an ARM SMMU by checking that
-	 * an ARM SMMU exists in the system before proceeding with the driver
-	 * and IOMMU bus operation registration.
-	 */
-	np = of_find_matching_node(NULL, arm_smmu_of_match);
-	if (!np)
-		return 0;
-
-	of_node_put(np);
 
 	ret = platform_driver_register(&arm_smmu_driver);
 	if (ret)
@@ -2002,28 +1992,8 @@ static void __exit arm_smmu_exit(void)
 {
 	return platform_driver_unregister(&arm_smmu_driver);
 }
-
-subsys_initcall(arm_smmu_init);
 module_exit(arm_smmu_exit);
 
-static int __init arm_smmu_of_init(struct device_node *np)
-{
-	struct arm_smmu_device *smmu;
-	struct platform_device *pdev;
-	int ret = arm_smmu_init();
-
-	if (ret)
-		return ret;
-
-	pdev = of_platform_device_create(np, NULL, platform_bus_type.dev_root);
-	if (!pdev)
-		return -ENODEV;
-
-	smmu = platform_get_drvdata(pdev);
-	of_iommu_set_ops(np, &arm_smmu_ops);
-
-	return 0;
-}
 IOMMU_OF_DECLARE(arm_smmuv1, "arm,smmu-v1", arm_smmu_of_init);
 IOMMU_OF_DECLARE(arm_smmuv2, "arm,smmu-v2", arm_smmu_of_init);
 IOMMU_OF_DECLARE(arm_mmu400, "arm,mmu-400", arm_smmu_of_init);
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

[RFC 9/9] drivers: iommu: arm-smmu: Set iommu_ops in probe

[Sricharan R]

iommu_ops should be set after the iommu is probed.
This ensures that the iommu is really ready when master's
iommu ops are set during their probe or else deferred.

Signed-off-by: Sricharan R <sricharan@codeaurora.org>
---
 drivers/iommu/arm-smmu.c | 34 ++--------------------------------
 1 file changed, 2 insertions(+), 32 deletions(-)

diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 2d65de4..f103d35 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1919,6 +1919,7 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 	if (of_property_read_bool(dev->of_node, "mmu-masters"))
 		arm_smmu_probe_mmu_masters(smmu);
 	arm_smmu_device_reset(smmu);
+	of_iommu_set_ops(dev->of_node, &arm_smmu_ops);
 	return 0;
 
 out_free_irqs:
@@ -1957,24 +1958,13 @@ static struct platform_driver arm_smmu_driver = {
 	.remove	= arm_smmu_device_remove,
 };
 
-static int __init arm_smmu_init(void)
+static int __init arm_smmu_of_init(struct device_node *np)
 {
-	struct device_node *np;
 	static bool done;
 	int ret;
 
 	if (done)
 		return 0;
-	/*
-	 * Play nice with systems that don't have an ARM SMMU by checking that
-	 * an ARM SMMU exists in the system before proceeding with the driver
-	 * and IOMMU bus operation registration.
-	 */
-	np = of_find_matching_node(NULL, arm_smmu_of_match);
-	if (!np)
-		return 0;
-
-	of_node_put(np);
 
 	ret = platform_driver_register(&arm_smmu_driver);
 	if (ret)
@@ -2002,28 +1992,8 @@ static void __exit arm_smmu_exit(void)
 {
 	return platform_driver_unregister(&arm_smmu_driver);
 }
-
-subsys_initcall(arm_smmu_init);
 module_exit(arm_smmu_exit);
 
-static int __init arm_smmu_of_init(struct device_node *np)
-{
-	struct arm_smmu_device *smmu;
-	struct platform_device *pdev;
-	int ret = arm_smmu_init();
-
-	if (ret)
-		return ret;
-
-	pdev = of_platform_device_create(np, NULL, platform_bus_type.dev_root);
-	if (!pdev)
-		return -ENODEV;
-
-	smmu = platform_get_drvdata(pdev);
-	of_iommu_set_ops(np, &arm_smmu_ops);
-
-	return 0;
-}
 IOMMU_OF_DECLARE(arm_smmuv1, "arm,smmu-v1", arm_smmu_of_init);
 IOMMU_OF_DECLARE(arm_smmuv2, "arm,smmu-v2", arm_smmu_of_init);
 IOMMU_OF_DECLARE(arm_mmu400, "arm,mmu-400", arm_smmu_of_init);
-- 
QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation

Re: [RFC 0/9] IOMMU probe deferral support

[Marek Szyprowski]

Hello,


On 2016-04-25 17:58, Sricharan R wrote:
> This is mostly a repost of the probe deferral series from
> Laurent Pinchart [1]. Added a check to fix boot with ACPI.
> Adapted arm-smmu driver to work with deferred probing and added
> a new api for the below reason. This is based on the generic iommu binding
> series from Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>.

Thanks for this patchset. I'm working on some serious rework in exynos power
domains and clocks support code and it turned out that I need this feature
to resolve probing order. It works fine on my internal tree, where some
iommu controllers cannot get their clocks early enough.

Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>

> Now associating a group with an master has become mandatory and happens
> when the master is added to the bus using BUS_ADD_DEVICE from iommu core.
> But the iommu has to be ready before this using early iommu registration
> and also master should have been added to the iommu using xlate. So, when
> trying to get rid of the early registration and using late probing for iommu
> devices to sort out the probing order, the newly added api is meant to be
> called during the late probing to add the master to the iommu. add_iommu_group
> could be modified to do the same though.
>
> The one issue here is the DRIVER_BIND notifier from iommu core
> might be called at a wrong point since the addition of a group to the device
> happens after this.
>
> Will be good to know the right direction to proceed on this fully.
>
> [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html
>
> Laurent Pinchart (6):
>    arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()
>    of: dma: Move range size workaround to of_dma_get_range()
>    of: dma: Make of_dma_deconfigure() public
>    of: dma: Split of_configure_dma() into mask and ops configuration
>    drivers: platform: Configure dma operations at probe time
>    iommu: of: Handle IOMMU lookup failure with deferred probing or error
>
> Sricharan R (3):
>    drivers: iommu: Add a new add device api
>    drivers: of: call iommu_bus_add_dev after iommu_configure_ops
>    drivers: iommu: arm-smmu: Set iommu_ops in probe
>
>   arch/arm/mm/dma-mapping.c |  9 ++++++
>   drivers/base/platform.c   | 13 ++++++++
>   drivers/iommu/arm-smmu.c  | 34 ++------------------
>   drivers/iommu/iommu.c     | 12 +++++++
>   drivers/iommu/of_iommu.c  | 16 ++++++++--
>   drivers/of/address.c      | 20 ++++++++++--
>   drivers/of/device.c       | 81 +++++++++++++++++++++++++++++++----------------
>   drivers/of/platform.c     | 16 ++++------
>   drivers/pci/probe.c       |  3 +-
>   include/linux/iommu.h     |  1 +
>   include/linux/of_device.h | 14 ++++++--
>   11 files changed, 141 insertions(+), 78 deletions(-)
>

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

[RFC 0/9] IOMMU probe deferral support

[Marek Szyprowski]

Hello,


On 2016-04-25 17:58, Sricharan R wrote:
> This is mostly a repost of the probe deferral series from
> Laurent Pinchart [1]. Added a check to fix boot with ACPI.
> Adapted arm-smmu driver to work with deferred probing and added
> a new api for the below reason. This is based on the generic iommu binding
> series from Robin Murphy <robin.murphy@arm.com>.

Thanks for this patchset. I'm working on some serious rework in exynos power
domains and clocks support code and it turned out that I need this feature
to resolve probing order. It works fine on my internal tree, where some
iommu controllers cannot get their clocks early enough.

Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>

> Now associating a group with an master has become mandatory and happens
> when the master is added to the bus using BUS_ADD_DEVICE from iommu core.
> But the iommu has to be ready before this using early iommu registration
> and also master should have been added to the iommu using xlate. So, when
> trying to get rid of the early registration and using late probing for iommu
> devices to sort out the probing order, the newly added api is meant to be
> called during the late probing to add the master to the iommu. add_iommu_group
> could be modified to do the same though.
>
> The one issue here is the DRIVER_BIND notifier from iommu core
> might be called at a wrong point since the addition of a group to the device
> happens after this.
>
> Will be good to know the right direction to proceed on this fully.
>
> [1] http://lists.linuxfoundation.org/pipermail/iommu/2015-May/013016.html
>
> Laurent Pinchart (6):
>    arm: dma-mapping: Don't override dma_ops in arch_setup_dma_ops()
>    of: dma: Move range size workaround to of_dma_get_range()
>    of: dma: Make of_dma_deconfigure() public
>    of: dma: Split of_configure_dma() into mask and ops configuration
>    drivers: platform: Configure dma operations at probe time
>    iommu: of: Handle IOMMU lookup failure with deferred probing or error
>
> Sricharan R (3):
>    drivers: iommu: Add a new add device api
>    drivers: of: call iommu_bus_add_dev after iommu_configure_ops
>    drivers: iommu: arm-smmu: Set iommu_ops in probe
>
>   arch/arm/mm/dma-mapping.c |  9 ++++++
>   drivers/base/platform.c   | 13 ++++++++
>   drivers/iommu/arm-smmu.c  | 34 ++------------------
>   drivers/iommu/iommu.c     | 12 +++++++
>   drivers/iommu/of_iommu.c  | 16 ++++++++--
>   drivers/of/address.c      | 20 ++++++++++--
>   drivers/of/device.c       | 81 +++++++++++++++++++++++++++++++----------------
>   drivers/of/platform.c     | 16 ++++------
>   drivers/pci/probe.c       |  3 +-
>   include/linux/iommu.h     |  1 +
>   include/linux/of_device.h | 14 ++++++--
>   11 files changed, 141 insertions(+), 78 deletions(-)
>

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

RE: [RFC 0/9] IOMMU probe deferral support

[Sricharan]

Hi Marek,

> -----Original Message-----
> From: linux-arm-kernel [mailto:linux-arm-kernel-
> bounces-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org] On Behalf Of Marek Szyprowski
> Sent: Thursday, May 12, 2016 6:23 PM
> To: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>; will.deacon-5wv7dgnIgG8@public.gmane.org;
> robin.murphy-5wv7dgnIgG8@public.gmane.org; joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org; iommu-cunTk1MwBs/ROKNJybVBZg@public.gmane.org
> foundation.org; linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org;
> laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org
> Subject: Re: [RFC 0/9] IOMMU probe deferral support
> 
> Hello,
> 
> 
> On 2016-04-25 17:58, Sricharan R wrote:
> > This is mostly a repost of the probe deferral series from Laurent
> > Pinchart [1]. Added a check to fix boot with ACPI.
> > Adapted arm-smmu driver to work with deferred probing and added a new
> > api for the below reason. This is based on the generic iommu binding
> > series from Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>.
> 
> Thanks for this patchset. I'm working on some serious rework in exynos
> power domains and clocks support code and it turned out that I need this
> feature to resolve probing order. It works fine on my internal tree, where
> some iommu controllers cannot get their clocks early enough.
> 
> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
> 
   Thanks for testing. So the issue that I was facing also was the same
where the iommu
   controllers cannot get clocks early. So waiting for some suggestions if
this is right/
   or there is another way for doing this probe deferral?

Regards,
 Sricharan

[RFC 0/9] IOMMU probe deferral support

[Sricharan]

Hi Marek,

> -----Original Message-----
> From: linux-arm-kernel [mailto:linux-arm-kernel-
> bounces at lists.infradead.org] On Behalf Of Marek Szyprowski
> Sent: Thursday, May 12, 2016 6:23 PM
> To: Sricharan R <sricharan@codeaurora.org>; will.deacon at arm.com;
> robin.murphy at arm.com; joro at 8bytes.org; iommu at lists.linux-
> foundation.org; linux-arm-kernel at lists.infradead.org;
> laurent.pinchart at ideasonboard.com
> Subject: Re: [RFC 0/9] IOMMU probe deferral support
> 
> Hello,
> 
> 
> On 2016-04-25 17:58, Sricharan R wrote:
> > This is mostly a repost of the probe deferral series from Laurent
> > Pinchart [1]. Added a check to fix boot with ACPI.
> > Adapted arm-smmu driver to work with deferred probing and added a new
> > api for the below reason. This is based on the generic iommu binding
> > series from Robin Murphy <robin.murphy@arm.com>.
> 
> Thanks for this patchset. I'm working on some serious rework in exynos
> power domains and clocks support code and it turned out that I need this
> feature to resolve probing order. It works fine on my internal tree, where
> some iommu controllers cannot get their clocks early enough.
> 
> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
> 
   Thanks for testing. So the issue that I was facing also was the same
where the iommu
   controllers cannot get clocks early. So waiting for some suggestions if
this is right/
   or there is another way for doing this probe deferral?

Regards,
 Sricharan

RE: [RFC 0/9] IOMMU probe deferral support

[Sricharan]

Hi Robin/Laurent,

>> -----Original Message-----
>> From: linux-arm-kernel [mailto:linux-arm-kernel-
>> bounces-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org] On Behalf Of Marek Szyprowski
>> Sent: Thursday, May 12, 2016 6:23 PM
>> To: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>; will.deacon-5wv7dgnIgG8@public.gmane.org;
>> robin.murphy-5wv7dgnIgG8@public.gmane.org; joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org; iommu-cunTk1MwBs/ROKNJybVBZg@public.gmane.org
>> foundation.org; linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org;
>> laurent.pinchart-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org
>> Subject: Re: [RFC 0/9] IOMMU probe deferral support
>>
>> Hello,
>>
>>
>> On 2016-04-25 17:58, Sricharan R wrote:
>> > This is mostly a repost of the probe deferral series from Laurent
>> > Pinchart [1]. Added a check to fix boot with ACPI.
>> > Adapted arm-smmu driver to work with deferred probing and added a new
>> > api for the below reason. This is based on the generic iommu binding
>> > series from Robin Murphy <robin.murphy-5wv7dgnIgG8@public.gmane.org>.
>>
>> Thanks for this patchset. I'm working on some serious rework in exynos
>> power domains and clocks support code and it turned out that I need this
>> feature to resolve probing order. It works fine on my internal tree, where
>> some iommu controllers cannot get their clocks early enough.
>>
>> Tested-by: Marek Szyprowski <m.szyprowski-Sze3O3UU22JBDgjK7y7TUQ@public.gmane.org>
>>
>   Thanks for testing. So the issue that I was facing also was the same where the iommu
>   controllers cannot get clocks early. So waiting for some suggestions if this is right/
>   or there is another way for doing this probe deferral?

So wanted to ask what will be approach to have the probe deferral working.
I remember that you mentioned last time that you were going to visit this. Hence thought
of asking how to proceed on this ?

Regards,
 Sricharan

[RFC 0/9] IOMMU probe deferral support

[Sricharan]

Hi Robin/Laurent,

>> -----Original Message-----
>> From: linux-arm-kernel [mailto:linux-arm-kernel-
>> bounces at lists.infradead.org] On Behalf Of Marek Szyprowski
>> Sent: Thursday, May 12, 2016 6:23 PM
>> To: Sricharan R <sricharan@codeaurora.org>; will.deacon at arm.com;
>> robin.murphy at arm.com; joro at 8bytes.org; iommu at lists.linux-
>> foundation.org; linux-arm-kernel at lists.infradead.org;
>> laurent.pinchart at ideasonboard.com
>> Subject: Re: [RFC 0/9] IOMMU probe deferral support
>>
>> Hello,
>>
>>
>> On 2016-04-25 17:58, Sricharan R wrote:
>> > This is mostly a repost of the probe deferral series from Laurent
>> > Pinchart [1]. Added a check to fix boot with ACPI.
>> > Adapted arm-smmu driver to work with deferred probing and added a new
>> > api for the below reason. This is based on the generic iommu binding
>> > series from Robin Murphy <robin.murphy@arm.com>.
>>
>> Thanks for this patchset. I'm working on some serious rework in exynos
>> power domains and clocks support code and it turned out that I need this
>> feature to resolve probing order. It works fine on my internal tree, where
>> some iommu controllers cannot get their clocks early enough.
>>
>> Tested-by: Marek Szyprowski <m.szyprowski@samsung.com>
>>
>   Thanks for testing. So the issue that I was facing also was the same where the iommu
>   controllers cannot get clocks early. So waiting for some suggestions if this is right/
>   or there is another way for doing this probe deferral?

So wanted to ask what will be approach to have the probe deferral working.
I remember that you mentioned last time that you were going to visit this. Hence thought
of asking how to proceed on this ?

Regards,
 Sricharan

Re: [RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Marek Szyprowski]

Hello,


On 2016-04-25 17:58, Sricharan R wrote:
> Now that the device's iommu ops are configured at probe time,
> the device has to be added to the iommu late.
>
> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
> ---
>   drivers/of/device.c | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index 57a5f2d..722115c 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -6,6 +6,7 @@
>   #include <linux/of_iommu.h>
>   #include <linux/dma-mapping.h>
>   #include <linux/init.h>
> +#include <linux/iommu.h>
>   #include <linux/module.h>
>   #include <linux/mod_devicetable.h>
>   #include <linux/slab.h>
> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>   		iommu ? " " : " not ");
>   
> +	if (iommu)
> +		iommu_bus_add_dev(dev);
> +

This causes build break when IOMMU subsystem is not enabled:

drivers/of/device.c: In function 'of_dma_configure_ops':
drivers/of/device.c:159:3: error: implicit declaration of function 
'iommu_bus_add_dev' [-Werror=implicit-function-declaration]
    iommu_bus_add_dev(dev);
    ^

>   	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
>   
>   	return 0;

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Marek Szyprowski]

Hello,


On 2016-04-25 17:58, Sricharan R wrote:
> Now that the device's iommu ops are configured at probe time,
> the device has to be added to the iommu late.
>
> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
> ---
>   drivers/of/device.c | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index 57a5f2d..722115c 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -6,6 +6,7 @@
>   #include <linux/of_iommu.h>
>   #include <linux/dma-mapping.h>
>   #include <linux/init.h>
> +#include <linux/iommu.h>
>   #include <linux/module.h>
>   #include <linux/mod_devicetable.h>
>   #include <linux/slab.h>
> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>   		iommu ? " " : " not ");
>   
> +	if (iommu)
> +		iommu_bus_add_dev(dev);
> +

This causes build break when IOMMU subsystem is not enabled:

drivers/of/device.c: In function 'of_dma_configure_ops':
drivers/of/device.c:159:3: error: implicit declaration of function 
'iommu_bus_add_dev' [-Werror=implicit-function-declaration]
    iommu_bus_add_dev(dev);
    ^

>   	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
>   
>   	return 0;

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland

RE: [RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan]

Hi Marek,

>On 2016-04-25 17:58, Sricharan R wrote:
>> Now that the device's iommu ops are configured at probe time,
>> the device has to be added to the iommu late.
>>
>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>> ---
>>   drivers/of/device.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index 57a5f2d..722115c 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -6,6 +6,7 @@
>>   #include <linux/of_iommu.h>
>>   #include <linux/dma-mapping.h>
>>   #include <linux/init.h>
>> +#include <linux/iommu.h>
>>   #include <linux/module.h>
>>   #include <linux/mod_devicetable.h>
>>   #include <linux/slab.h>
>> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>>   		iommu ? " " : " not ");
>>
>> +	if (iommu)
>> +		iommu_bus_add_dev(dev);
>> +
>
>This causes build break when IOMMU subsystem is not enabled:
>
>drivers/of/device.c: In function 'of_dma_configure_ops':
>drivers/of/device.c:159:3: error: implicit declaration of function
>'iommu_bus_add_dev' [-Werror=implicit-function-declaration]
>    iommu_bus_add_dev(dev);
>    ^

Ah ok. Thanks for pointing this out. I will repost, with this fixed.
I also realised that PATCH 9, might cause a issue in NON-DT
builds and needs a correction. But otherwise, i am waiting for
suggestions/feedback on how to proceed with this series.

Regards,
 Sricharan

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan]

Hi Marek,

>On 2016-04-25 17:58, Sricharan R wrote:
>> Now that the device's iommu ops are configured at probe time,
>> the device has to be added to the iommu late.
>>
>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>> ---
>>   drivers/of/device.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index 57a5f2d..722115c 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -6,6 +6,7 @@
>>   #include <linux/of_iommu.h>
>>   #include <linux/dma-mapping.h>
>>   #include <linux/init.h>
>> +#include <linux/iommu.h>
>>   #include <linux/module.h>
>>   #include <linux/mod_devicetable.h>
>>   #include <linux/slab.h>
>> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>>   		iommu ? " " : " not ");
>>
>> +	if (iommu)
>> +		iommu_bus_add_dev(dev);
>> +
>
>This causes build break when IOMMU subsystem is not enabled:
>
>drivers/of/device.c: In function 'of_dma_configure_ops':
>drivers/of/device.c:159:3: error: implicit declaration of function
>'iommu_bus_add_dev' [-Werror=implicit-function-declaration]
>    iommu_bus_add_dev(dev);
>    ^

Ah ok. Thanks for pointing this out. I will repost, with this fixed.
I also realised that PATCH 9, might cause a issue in NON-DT
builds and needs a correction. But otherwise, i am waiting for
suggestions/feedback on how to proceed with this series.

Regards,
 Sricharan

Re: [RFC 5/9] drivers: platform: Configure dma operations at probe time

[Robin Murphy]

On 25/04/16 16:58, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>
> Configuring DMA ops at probe time will allow deferring device probe when
> the IOMMU isn't available yet.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
> ---
>   drivers/base/platform.c | 13 +++++++++++++
>   drivers/of/platform.c   |  7 +++----
>   2 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/base/platform.c b/drivers/base/platform.c
> index f437afa..17e64a4 100644
> --- a/drivers/base/platform.c
> +++ b/drivers/base/platform.c
> @@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
>   	if (ret < 0)
>   		return ret;
>
> +	if (of_have_populated_dt()) {
> +		ret = of_dma_configure_ops(_dev, _dev->of_node);
> +		if (ret < 0)
> +			goto done;
> +	}
> +
>   	ret = dev_pm_domain_attach(_dev, true);
>   	if (ret != -EPROBE_DEFER) {
>   		if (drv->probe) {
> @@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
>   		}
>   	}
>
> +	if (of_have_populated_dt()) {
> +		if (ret)
> +			of_dma_deconfigure(_dev);
> +	}
> +done:
>   	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
>   		dev_warn(_dev, "probe deferral not supported\n");
>   		ret = -ENXIO;
> @@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
>   	if (drv->remove)
>   		ret = drv->remove(dev);
>   	dev_pm_domain_detach(_dev, true);
> +	if (of_have_populated_dt())
> +		of_dma_deconfigure(_dev);
>
>   	return ret;
>   }
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 17ee8d5..12bbc8e1 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
>   	dev->dev.bus = &platform_bus_type;
>   	dev->dev.platform_data = platform_data;
>   	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
> -	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
>   	of_msi_configure(&dev->dev, dev->dev.of_node);
>
>   	if (of_device_add(dev) != 0) {
> -		of_dma_deconfigure(&dev->dev);
>   		platform_device_put(dev);
>   		goto err_clear_flag;
>   	}
> @@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>   	if (dev->bus == &platform_bus_type)
>   		platform_device_unregister(to_platform_device(dev));
>   #ifdef CONFIG_ARM_AMBA
> -	else if (dev->bus == &amba_bustype)
> +	else if (dev->bus == &amba_bustype) {
>   		amba_device_unregister(to_amba_device(dev));
> +		of_dma_deconfigure(dev);
> +	}

Note that we definitely need this to happen properly on other buses, 
too. I had something for the AMBA bus back when I last looked at 
this[0], but since then we now have PCI devices going through the 
of_dma_configure() path as well (hence 226d89cbb242).

There's also now some initial ACPI code in flight[1], so it's probably 
about time to take another closer look at how this can be properly 
generalised, but I'm not necessarily averse to starting with a 
DT-focused solution that gets at least half way, then building on top of 
that.

Robin.

[0]:http://www.linux-arm.org/git?p=linux-rm.git;a=commitdiff;h=1f38b62b05e17ccbfcc68e4229c69fee74922e2a
[1]:http://thread.gmane.org/gmane.linux.kernel.iommu/13046

>   #endif
>
> -	of_dma_deconfigure(dev);
>   	of_node_clear_flag(dev->of_node, OF_POPULATED);
>   	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
>   	return 0;
>

[RFC 5/9] drivers: platform: Configure dma operations at probe time

[Robin Murphy]

On 25/04/16 16:58, Sricharan R wrote:
> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>
> Configuring DMA ops at probe time will allow deferring device probe when
> the IOMMU isn't available yet.
>
> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
> ---
>   drivers/base/platform.c | 13 +++++++++++++
>   drivers/of/platform.c   |  7 +++----
>   2 files changed, 16 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/base/platform.c b/drivers/base/platform.c
> index f437afa..17e64a4 100644
> --- a/drivers/base/platform.c
> +++ b/drivers/base/platform.c
> @@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
>   	if (ret < 0)
>   		return ret;
>
> +	if (of_have_populated_dt()) {
> +		ret = of_dma_configure_ops(_dev, _dev->of_node);
> +		if (ret < 0)
> +			goto done;
> +	}
> +
>   	ret = dev_pm_domain_attach(_dev, true);
>   	if (ret != -EPROBE_DEFER) {
>   		if (drv->probe) {
> @@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
>   		}
>   	}
>
> +	if (of_have_populated_dt()) {
> +		if (ret)
> +			of_dma_deconfigure(_dev);
> +	}
> +done:
>   	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
>   		dev_warn(_dev, "probe deferral not supported\n");
>   		ret = -ENXIO;
> @@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
>   	if (drv->remove)
>   		ret = drv->remove(dev);
>   	dev_pm_domain_detach(_dev, true);
> +	if (of_have_populated_dt())
> +		of_dma_deconfigure(_dev);
>
>   	return ret;
>   }
> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
> index 17ee8d5..12bbc8e1 100644
> --- a/drivers/of/platform.c
> +++ b/drivers/of/platform.c
> @@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
>   	dev->dev.bus = &platform_bus_type;
>   	dev->dev.platform_data = platform_data;
>   	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
> -	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
>   	of_msi_configure(&dev->dev, dev->dev.of_node);
>
>   	if (of_device_add(dev) != 0) {
> -		of_dma_deconfigure(&dev->dev);
>   		platform_device_put(dev);
>   		goto err_clear_flag;
>   	}
> @@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>   	if (dev->bus == &platform_bus_type)
>   		platform_device_unregister(to_platform_device(dev));
>   #ifdef CONFIG_ARM_AMBA
> -	else if (dev->bus == &amba_bustype)
> +	else if (dev->bus == &amba_bustype) {
>   		amba_device_unregister(to_amba_device(dev));
> +		of_dma_deconfigure(dev);
> +	}

Note that we definitely need this to happen properly on other buses, 
too. I had something for the AMBA bus back when I last looked at 
this[0], but since then we now have PCI devices going through the 
of_dma_configure() path as well (hence 226d89cbb242).

There's also now some initial ACPI code in flight[1], so it's probably 
about time to take another closer look at how this can be properly 
generalised, but I'm not necessarily averse to starting with a 
DT-focused solution that gets at least half way, then building on top of 
that.

Robin.

[0]:http://www.linux-arm.org/git?p=linux-rm.git;a=commitdiff;h=1f38b62b05e17ccbfcc68e4229c69fee74922e2a
[1]:http://thread.gmane.org/gmane.linux.kernel.iommu/13046

>   #endif
>
> -	of_dma_deconfigure(dev);
>   	of_node_clear_flag(dev->of_node, OF_POPULATED);
>   	of_node_clear_flag(dev->of_node, OF_POPULATED_BUS);
>   	return 0;
>

Re: [RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Robin Murphy]

On 25/04/16 16:58, Sricharan R wrote:
> Now that the device's iommu ops are configured at probe time,
> the device has to be added to the iommu late.
>
> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
> ---
>   drivers/of/device.c | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index 57a5f2d..722115c 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -6,6 +6,7 @@
>   #include <linux/of_iommu.h>
>   #include <linux/dma-mapping.h>
>   #include <linux/init.h>
> +#include <linux/iommu.h>
>   #include <linux/module.h>
>   #include <linux/mod_devicetable.h>
>   #include <linux/slab.h>
> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>   		iommu ? " " : " not ");
>
> +	if (iommu)
> +		iommu_bus_add_dev(dev);

This (in conjunction with the previous patch) seems unnecessarily 
convoluted - if of_iommu_configure() has found some iommu_ops for a 
device, why not just call .add_device() directly there and then? There 
are already systems that could warrant having two different IOMMU 
drivers active simultaneously (but thankfully don't _need_ to), so 
trying to escape from per-bus IOMMU ops makes more sense than 
entrenching the horrible notion of "the" IOMMU on "the" platform bus any 
further.

Robin.

> +
>   	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
>
>   	return 0;
>

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Robin Murphy]

On 25/04/16 16:58, Sricharan R wrote:
> Now that the device's iommu ops are configured at probe time,
> the device has to be added to the iommu late.
>
> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
> ---
>   drivers/of/device.c | 4 ++++
>   1 file changed, 4 insertions(+)
>
> diff --git a/drivers/of/device.c b/drivers/of/device.c
> index 57a5f2d..722115c 100644
> --- a/drivers/of/device.c
> +++ b/drivers/of/device.c
> @@ -6,6 +6,7 @@
>   #include <linux/of_iommu.h>
>   #include <linux/dma-mapping.h>
>   #include <linux/init.h>
> +#include <linux/iommu.h>
>   #include <linux/module.h>
>   #include <linux/mod_devicetable.h>
>   #include <linux/slab.h>
> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>   		iommu ? " " : " not ");
>
> +	if (iommu)
> +		iommu_bus_add_dev(dev);

This (in conjunction with the previous patch) seems unnecessarily 
convoluted - if of_iommu_configure() has found some iommu_ops for a 
device, why not just call .add_device() directly there and then? There 
are already systems that could warrant having two different IOMMU 
drivers active simultaneously (but thankfully don't _need_ to), so 
trying to escape from per-bus IOMMU ops makes more sense than 
entrenching the horrible notion of "the" IOMMU on "the" platform bus any 
further.

Robin.

> +
>   	arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
>
>   	return 0;
>

RE: [RFC 5/9] drivers: platform: Configure dma operations at probe time

[Sricharan]

Hi Robin,

>On 25/04/16 16:58, Sricharan R wrote:
>> From: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>>
>> Configuring DMA ops at probe time will allow deferring device probe when
>> the IOMMU isn't available yet.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas-ryLnwIuWjnjg/C1BVhZhaw@public.gmane.org>
>> ---
>>   drivers/base/platform.c | 13 +++++++++++++
>>   drivers/of/platform.c   |  7 +++----
>>   2 files changed, 16 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/base/platform.c b/drivers/base/platform.c
>> index f437afa..17e64a4 100644
>> --- a/drivers/base/platform.c
>> +++ b/drivers/base/platform.c
>> @@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
>>   	if (ret < 0)
>>   		return ret;
>>
>> +	if (of_have_populated_dt()) {
>> +		ret = of_dma_configure_ops(_dev, _dev->of_node);
>> +		if (ret < 0)
>> +			goto done;
>> +	}
>> +
>>   	ret = dev_pm_domain_attach(_dev, true);
>>   	if (ret != -EPROBE_DEFER) {
>>   		if (drv->probe) {
>> @@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
>>   		}
>>   	}
>>
>> +	if (of_have_populated_dt()) {
>> +		if (ret)
>> +			of_dma_deconfigure(_dev);
>> +	}
>> +done:
>>   	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
>>   		dev_warn(_dev, "probe deferral not supported\n");
>>   		ret = -ENXIO;
>> @@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
>>   	if (drv->remove)
>>   		ret = drv->remove(dev);
>>   	dev_pm_domain_detach(_dev, true);
>> +	if (of_have_populated_dt())
>> +		of_dma_deconfigure(_dev);
>>
>>   	return ret;
>>   }
>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
>> index 17ee8d5..12bbc8e1 100644
>> --- a/drivers/of/platform.c
>> +++ b/drivers/of/platform.c
>> @@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
>>   	dev->dev.bus = &platform_bus_type;
>>   	dev->dev.platform_data = platform_data;
>>   	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
>> -	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
>>   	of_msi_configure(&dev->dev, dev->dev.of_node);
>>
>>   	if (of_device_add(dev) != 0) {
>> -		of_dma_deconfigure(&dev->dev);
>>   		platform_device_put(dev);
>>   		goto err_clear_flag;
>>   	}
>> @@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>>   	if (dev->bus == &platform_bus_type)
>>   		platform_device_unregister(to_platform_device(dev));
>>   #ifdef CONFIG_ARM_AMBA
>> -	else if (dev->bus == &amba_bustype)
>> +	else if (dev->bus == &amba_bustype) {
>>   		amba_device_unregister(to_amba_device(dev));
>> +		of_dma_deconfigure(dev);
>> +	}
>
>Note that we definitely need this to happen properly on other buses,
>too. I had something for the AMBA bus back when I last looked at
>this[0], but since then we now have PCI devices going through the
>of_dma_configure() path as well (hence 226d89cbb242).
>
>There's also now some initial ACPI code in flight[1], so it's probably
>about time to take another closer look at how this can be properly
>generalised, but I'm not necessarily averse to starting with a
>DT-focused solution that gets at least half way, then building on top of
>that.

Thanks for direction. I will definitely look at how this can be generalised
 for other buses as well. Based on how that looks, as you said if required
 a DT-only can be done first.

Regards,
 Sricharan

[RFC 5/9] drivers: platform: Configure dma operations at probe time

[Sricharan]

Hi Robin,

>On 25/04/16 16:58, Sricharan R wrote:
>> From: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>>
>> Configuring DMA ops at probe time will allow deferring device probe when
>> the IOMMU isn't available yet.
>>
>> Signed-off-by: Laurent Pinchart <laurent.pinchart+renesas@ideasonboard.com>
>> ---
>>   drivers/base/platform.c | 13 +++++++++++++
>>   drivers/of/platform.c   |  7 +++----
>>   2 files changed, 16 insertions(+), 4 deletions(-)
>>
>> diff --git a/drivers/base/platform.c b/drivers/base/platform.c
>> index f437afa..17e64a4 100644
>> --- a/drivers/base/platform.c
>> +++ b/drivers/base/platform.c
>> @@ -557,6 +557,12 @@ static int platform_drv_probe(struct device *_dev)
>>   	if (ret < 0)
>>   		return ret;
>>
>> +	if (of_have_populated_dt()) {
>> +		ret = of_dma_configure_ops(_dev, _dev->of_node);
>> +		if (ret < 0)
>> +			goto done;
>> +	}
>> +
>>   	ret = dev_pm_domain_attach(_dev, true);
>>   	if (ret != -EPROBE_DEFER) {
>>   		if (drv->probe) {
>> @@ -569,6 +575,11 @@ static int platform_drv_probe(struct device *_dev)
>>   		}
>>   	}
>>
>> +	if (of_have_populated_dt()) {
>> +		if (ret)
>> +			of_dma_deconfigure(_dev);
>> +	}
>> +done:
>>   	if (drv->prevent_deferred_probe && ret == -EPROBE_DEFER) {
>>   		dev_warn(_dev, "probe deferral not supported\n");
>>   		ret = -ENXIO;
>> @@ -591,6 +602,8 @@ static int platform_drv_remove(struct device *_dev)
>>   	if (drv->remove)
>>   		ret = drv->remove(dev);
>>   	dev_pm_domain_detach(_dev, true);
>> +	if (of_have_populated_dt())
>> +		of_dma_deconfigure(_dev);
>>
>>   	return ret;
>>   }
>> diff --git a/drivers/of/platform.c b/drivers/of/platform.c
>> index 17ee8d5..12bbc8e1 100644
>> --- a/drivers/of/platform.c
>> +++ b/drivers/of/platform.c
>> @@ -180,11 +180,9 @@ static struct platform_device *of_platform_device_create_pdata(
>>   	dev->dev.bus = &platform_bus_type;
>>   	dev->dev.platform_data = platform_data;
>>   	of_dma_configure_masks(&dev->dev, dev->dev.of_node);
>> -	of_dma_configure_ops(&dev->dev, dev->dev.of_node);
>>   	of_msi_configure(&dev->dev, dev->dev.of_node);
>>
>>   	if (of_device_add(dev) != 0) {
>> -		of_dma_deconfigure(&dev->dev);
>>   		platform_device_put(dev);
>>   		goto err_clear_flag;
>>   	}
>> @@ -481,11 +479,12 @@ static int of_platform_device_destroy(struct device *dev, void *data)
>>   	if (dev->bus == &platform_bus_type)
>>   		platform_device_unregister(to_platform_device(dev));
>>   #ifdef CONFIG_ARM_AMBA
>> -	else if (dev->bus == &amba_bustype)
>> +	else if (dev->bus == &amba_bustype) {
>>   		amba_device_unregister(to_amba_device(dev));
>> +		of_dma_deconfigure(dev);
>> +	}
>
>Note that we definitely need this to happen properly on other buses,
>too. I had something for the AMBA bus back when I last looked at
>this[0], but since then we now have PCI devices going through the
>of_dma_configure() path as well (hence 226d89cbb242).
>
>There's also now some initial ACPI code in flight[1], so it's probably
>about time to take another closer look at how this can be properly
>generalised, but I'm not necessarily averse to starting with a
>DT-focused solution that gets at least half way, then building on top of
>that.

Thanks for direction. I will definitely look at how this can be generalised
 for other buses as well. Based on how that looks, as you said if required
 a DT-only can be done first.

Regards,
 Sricharan

RE: [RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan]

Hi Robin,

>On 25/04/16 16:58, Sricharan R wrote:
>> Now that the device's iommu ops are configured at probe time,
>> the device has to be added to the iommu late.
>>
>> Signed-off-by: Sricharan R <sricharan-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
>> ---
>>   drivers/of/device.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index 57a5f2d..722115c 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -6,6 +6,7 @@
>>   #include <linux/of_iommu.h>
>>   #include <linux/dma-mapping.h>
>>   #include <linux/init.h>
>> +#include <linux/iommu.h>
>>   #include <linux/module.h>
>>   #include <linux/mod_devicetable.h>
>>   #include <linux/slab.h>
>> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>>   		iommu ? " " : " not ");
>>
>> +	if (iommu)
>> +		iommu_bus_add_dev(dev);
>
>This (in conjunction with the previous patch) seems unnecessarily
>convoluted - if of_iommu_configure() has found some iommu_ops for a
>device, why not just call .add_device() directly there and then? There
>are already systems that could warrant having two different IOMMU
>drivers active simultaneously (but thankfully don't _need_ to), so
>trying to escape from per-bus IOMMU ops makes more sense than
>entrenching the horrible notion of "the" IOMMU on "the" platform bus any
>further.
>
  Ok, agree on this. This will remove the addition of that api in previous
  patch as well. Will change this.

Regards,
 Sricharan

[RFC 8/9] drivers: of: call iommu_bus_add_dev after iommu_configure_ops

[Sricharan]

Hi Robin,

>On 25/04/16 16:58, Sricharan R wrote:
>> Now that the device's iommu ops are configured at probe time,
>> the device has to be added to the iommu late.
>>
>> Signed-off-by: Sricharan R <sricharan@codeaurora.org>
>> ---
>>   drivers/of/device.c | 4 ++++
>>   1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/of/device.c b/drivers/of/device.c
>> index 57a5f2d..722115c 100644
>> --- a/drivers/of/device.c
>> +++ b/drivers/of/device.c
>> @@ -6,6 +6,7 @@
>>   #include <linux/of_iommu.h>
>>   #include <linux/dma-mapping.h>
>>   #include <linux/init.h>
>> +#include <linux/iommu.h>
>>   #include <linux/module.h>
>>   #include <linux/mod_devicetable.h>
>>   #include <linux/slab.h>
>> @@ -154,6 +155,9 @@ int of_dma_configure_ops(struct device *dev, struct device_node *np)
>>   	dev_dbg(dev, "device is%sbehind an iommu\n",
>>   		iommu ? " " : " not ");
>>
>> +	if (iommu)
>> +		iommu_bus_add_dev(dev);
>
>This (in conjunction with the previous patch) seems unnecessarily
>convoluted - if of_iommu_configure() has found some iommu_ops for a
>device, why not just call .add_device() directly there and then? There
>are already systems that could warrant having two different IOMMU
>drivers active simultaneously (but thankfully don't _need_ to), so
>trying to escape from per-bus IOMMU ops makes more sense than
>entrenching the horrible notion of "the" IOMMU on "the" platform bus any
>further.
>
  Ok, agree on this. This will remove the addition of that api in previous
  patch as well. Will change this.

Regards,
 Sricharan