dmaengine Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH 0/3] dmaengine: Support for DMA domain controllers
@ 2019-09-10 11:50 Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains Peter Ujfalusi
                   ` (2 more replies)
  0 siblings, 3 replies; 6+ messages in thread
From: Peter Ujfalusi @ 2019-09-10 11:50 UTC (permalink / raw)
  To: vkoul, robh+dt; +Cc: dmaengine, linux-kernel, dan.j.williams, devicetree

Hi,

Changes since RFC:
- Extended the binding document's example
- Changed the API to dma_request_chan_by_domain(dev, mask)
 - Fixed certain crash if the dev parameter would be NULL
- Add missing parenthesis in of_dma.c
- typo fix.

More and more SoC have more than one DMA controller integrated.

If a device needs none slave DMA channel for operation (block copy from/to
memory mapped regions for example) at the moment when they request a channel it
is going to be taken from the first DMA controller which was registered, but
this might be not optimal for the device.

For example on AM654 we have two DMAs: main_udmap and mcu_udmap.
DDR to DDR memcpy is twice as fast on main_udmap compared to mcu_udmap, while
devices on MCU domain (OSPI for example) are more than twice as fast on
mcu_udmap than with main_udmap.

Because of probing order (mcu_udmap is probing first) modules would use
mcu_udmap instead of the better main_udmap. Currently the only solution is to
make a choice and disable the MEM_TO_MEM functionality on one of them which is
not a great solution.

With the introduction of DMA domain controllers we can utilize the best DMA
controller for the job around the SoC without the need to degrade performance.

If the dma-domain-controller is not present in DT or booted w/o DT the none
slave channel request will work as it does today.

Regards,
Peter
---
Peter Ujfalusi (3):
  dt-bindings: dma: Add documentation for DMA domains
  dmaengine: of_dma: Function to look up the DMA domain of a client
  dmaengine: Support for requesting channels preferring DMA domain
    controller

 .../devicetree/bindings/dma/dma-domain.yaml   | 88 +++++++++++++++++++
 drivers/dma/dmaengine.c                       | 21 +++--
 drivers/dma/of-dma.c                          | 42 +++++++++
 include/linux/dmaengine.h                     |  9 +-
 include/linux/of_dma.h                        |  7 ++
 5 files changed, 159 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml

-- 
Peter

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains
  2019-09-10 11:50 [PATCH 0/3] dmaengine: Support for DMA domain controllers Peter Ujfalusi
@ 2019-09-10 11:50 ` Peter Ujfalusi
  2019-09-13 14:36   ` Rob Herring
  2019-09-10 11:50 ` [PATCH 2/3] dmaengine: of_dma: Function to look up the DMA domain of a client Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 3/3] dmaengine: Support for requesting channels preferring DMA domain controller Peter Ujfalusi
  2 siblings, 1 reply; 6+ messages in thread
From: Peter Ujfalusi @ 2019-09-10 11:50 UTC (permalink / raw)
  To: vkoul, robh+dt; +Cc: dmaengine, linux-kernel, dan.j.williams, devicetree

On systems where multiple DMA controllers available, non Slave (for example
memcpy operation) users can not be described in DT as there is no device
involved from the DMA controller's point of view, DMA binding is not usable.
However in these systems still a peripheral might need to be serviced by or
it is better to serviced by specific DMA controller.
When a memcpy is used to/from a memory mapped region for example a DMA in the
same domain can perform better.
For generic software modules doing mem 2 mem operations it also matter that
they will get a channel from a controller which is faster in DDR to DDR mode
rather then from the first controller happen to be loaded.

This property is inherited, so it may be specified in a device node or in any
of its parent nodes.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
---
 .../devicetree/bindings/dma/dma-domain.yaml   | 88 +++++++++++++++++++
 1 file changed, 88 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml

diff --git a/Documentation/devicetree/bindings/dma/dma-domain.yaml b/Documentation/devicetree/bindings/dma/dma-domain.yaml
new file mode 100644
index 000000000000..da59bb129c58
--- /dev/null
+++ b/Documentation/devicetree/bindings/dma/dma-domain.yaml
@@ -0,0 +1,88 @@
+# SPDX-License-Identifier: GPL-2.0
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/dma/dma-controller.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: DMA Domain Controller Definition
+
+maintainers:
+  - Vinod Koul <vkoul@kernel.org>
+
+allOf:
+  - $ref: "dma-controller.yaml#"
+
+description:
+  On systems where multiple DMA controllers available, none Slave (for example
+  memcpy operation) users can not be described in DT as there is no device
+  involved from the DMA controller's point of view, DMA binding is not usable.
+  However in these systems still a peripheral might need to be serviced by or
+  it is better to serviced by specific DMA controller.
+  When a memcpy is used to/from a memory mapped region for example a DMA in the
+  same domain can perform better.
+  For generic software modules doing mem 2 mem operations it also matter that
+  they will get a channel from a controller which is faster in DDR to DDR mode
+  rather then from the first controller happen to be loaded.
+
+  This property is inherited, so it may be specified in a device node or in any
+  of its parent nodes.
+
+properties:
+  $dma-domain-controller:
+    $ref: /schemas/types.yaml#definitions/phandle
+    description:
+      phande to the DMA controller node which should be used for the device or
+      domain.
+
+examples:
+  - |
+    / {
+        model = "Texas Instruments K3 AM654 SoC";
+        compatible = "ti,am654";
+        interrupt-parent = <&gic500>;
+        /* For modules without device, DDR to DDR is faster on main UDMAP */
+        dma-domain-controller = <&main_udmap>;
+        #address-cells = <2>;
+        #size-cells = <2>;
+        ...
+    };
+
+    &cbass_main {
+        /* For modules within MAIN domain, use main UDMAP */
+        dma-domain-controller = <&main_udmap>;
+
+        cbass_main_navss: interconnect0 {
+            ...
+            main_udmap: dma-controller@31150000 {
+                compatible = "ti,am654-navss-main-udmap";
+                ...
+            };
+        };
+    };
+
+    &cbass_mcu {
+        /* For modules within MCU domain, use mcu UDMAP */
+        dma-domain-controller = <&mcu_udmap>;
+
+        cbass_mcu_navss: interconnect1 {
+            ...
+            mcu_udmap: dma-controller@285c0000 {
+                compatible = "ti,am654-navss-mcu-udmap";
+                ...
+            };
+        };
+
+        fss: fss@47000000 {
+            compatible = "simple-bus";
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+
+            ospi0: spi@47040000 {
+                compatible = "ti,am654-ospi", "cdns,qspi-nor";
+                ...
+                /* memcpy channel will be request from mcu_udmap */
+            };
+        };
+    };
+...
-- 
Peter

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 2/3] dmaengine: of_dma: Function to look up the DMA domain of a client
  2019-09-10 11:50 [PATCH 0/3] dmaengine: Support for DMA domain controllers Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains Peter Ujfalusi
@ 2019-09-10 11:50 ` Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 3/3] dmaengine: Support for requesting channels preferring DMA domain controller Peter Ujfalusi
  2 siblings, 0 replies; 6+ messages in thread
From: Peter Ujfalusi @ 2019-09-10 11:50 UTC (permalink / raw)
  To: vkoul, robh+dt; +Cc: dmaengine, linux-kernel, dan.j.williams, devicetree

Find the DMA domain controller of the client device by iterating up in
device tree looking for the closest 'dma-domain-controller' property.

If the client's node is not provided then check the DT root for the
controller.

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
---
 drivers/dma/of-dma.c   | 42 ++++++++++++++++++++++++++++++++++++++++++
 include/linux/of_dma.h |  7 +++++++
 2 files changed, 49 insertions(+)

diff --git a/drivers/dma/of-dma.c b/drivers/dma/of-dma.c
index c2d779daa4b5..b4bff47006b0 100644
--- a/drivers/dma/of-dma.c
+++ b/drivers/dma/of-dma.c
@@ -18,6 +18,48 @@
 static LIST_HEAD(of_dma_list);
 static DEFINE_MUTEX(of_dma_lock);
 
+/**
+ * of_find_dma_domain - Get the domain DMA controller
+ * @np:		device node of the client device
+ *
+ * Look up the DMA controller of the domain the client device is part of.
+ * Finds the dma-domain controller the client device belongs to. It is used when
+ * requesting non slave channels (like channel for memcpy) to make sure that the
+ * channel can be request from a DMA controller which can service the given
+ * domain best.
+ *
+ * Returns the device_node pointer of the DMA controller or succes or NULL on
+ * error.
+ */
+struct device_node *of_find_dma_domain(struct device_node *np)
+{
+	struct device_node *dma_domain = NULL;
+	phandle dma_phandle;
+
+	/*
+	 * If no device_node is provided look at the root level for system
+	 * default DMA controller for modules.
+	 */
+	if (!np)
+		np = of_root;
+
+	if (!np || !of_node_get(np))
+		return NULL;
+
+	do {
+		if (of_property_read_u32(np, "dma-domain-controller",
+					 &dma_phandle)) {
+			np = of_get_next_parent(np);
+		} else {
+			dma_domain = of_find_node_by_phandle(dma_phandle);
+			of_node_put(np);
+		}
+	} while (!dma_domain && np);
+
+	return dma_domain;
+}
+EXPORT_SYMBOL_GPL(of_find_dma_domain);
+
 /**
  * of_dma_find_controller - Get a DMA controller in DT DMA helpers list
  * @dma_spec:	pointer to DMA specifier as found in the device tree
diff --git a/include/linux/of_dma.h b/include/linux/of_dma.h
index fd706cdf255c..6eab0a8d3335 100644
--- a/include/linux/of_dma.h
+++ b/include/linux/of_dma.h
@@ -32,6 +32,8 @@ struct of_dma_filter_info {
 };
 
 #ifdef CONFIG_DMA_OF
+extern struct device_node *of_find_dma_domain(struct device_node *np);
+
 extern int of_dma_controller_register(struct device_node *np,
 		struct dma_chan *(*of_dma_xlate)
 		(struct of_phandle_args *, struct of_dma *),
@@ -52,6 +54,11 @@ extern struct dma_chan *of_dma_xlate_by_chan_id(struct of_phandle_args *dma_spec
 		struct of_dma *ofdma);
 
 #else
+static inline struct device_node *of_find_dma_domain(struct device_node *np)
+{
+	return NULL;
+}
+
 static inline int of_dma_controller_register(struct device_node *np,
 		struct dma_chan *(*of_dma_xlate)
 		(struct of_phandle_args *, struct of_dma *),
-- 
Peter

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH 3/3] dmaengine: Support for requesting channels preferring DMA domain controller
  2019-09-10 11:50 [PATCH 0/3] dmaengine: Support for DMA domain controllers Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains Peter Ujfalusi
  2019-09-10 11:50 ` [PATCH 2/3] dmaengine: of_dma: Function to look up the DMA domain of a client Peter Ujfalusi
@ 2019-09-10 11:50 ` Peter Ujfalusi
  2 siblings, 0 replies; 6+ messages in thread
From: Peter Ujfalusi @ 2019-09-10 11:50 UTC (permalink / raw)
  To: vkoul, robh+dt; +Cc: dmaengine, linux-kernel, dan.j.williams, devicetree

In case the channel is not requested via the slave API, use the
of_find_dma_domain() to see if a system default DMA controller is
specified.

Add new function which can be used by clients to request channels by mask
from their DMA domain controller if specified.

Client drivers can take advantage of the domain support by moving from
dma_request_chan_by_mask() to dma_request_chan_by_domain()

Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
---
 drivers/dma/dmaengine.c   | 21 ++++++++++++++++-----
 include/linux/dmaengine.h |  9 ++++++---
 2 files changed, 22 insertions(+), 8 deletions(-)

diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index 03ac4b96117c..1bae3ff24da0 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -640,6 +640,10 @@ struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
 	struct dma_device *device, *_d;
 	struct dma_chan *chan = NULL;
 
+	/* If np is not specified, get the default DMA domain controller */
+	if (!np)
+		np = of_find_dma_domain(NULL);
+
 	/* Find a channel */
 	mutex_lock(&dma_list_mutex);
 	list_for_each_entry_safe(device, _d, &dma_device_list, global_node) {
@@ -751,19 +755,26 @@ struct dma_chan *dma_request_slave_channel(struct device *dev,
 EXPORT_SYMBOL_GPL(dma_request_slave_channel);
 
 /**
- * dma_request_chan_by_mask - allocate a channel satisfying certain capabilities
- * @mask: capabilities that the channel must satisfy
+ * dma_request_chan_by_domain - allocate a channel by mask from DMA domain
+ * @dev:	pointer to client device structure
+ * @mask:	capabilities that the channel must satisfy
  *
  * Returns pointer to appropriate DMA channel on success or an error pointer.
  */
-struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask)
+struct dma_chan *dma_request_chan_by_domain(struct device *dev,
+					    const dma_cap_mask_t *mask)
 {
 	struct dma_chan *chan;
 
 	if (!mask)
 		return ERR_PTR(-ENODEV);
 
-	chan = __dma_request_channel(mask, NULL, NULL, NULL);
+	if (dev)
+		chan = __dma_request_channel(mask, NULL, NULL,
+					     of_find_dma_domain(dev->of_node));
+	else
+		chan = __dma_request_channel(mask, NULL, NULL, NULL);
+
 	if (!chan) {
 		mutex_lock(&dma_list_mutex);
 		if (list_empty(&dma_device_list))
@@ -775,7 +786,7 @@ struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask)
 
 	return chan;
 }
-EXPORT_SYMBOL_GPL(dma_request_chan_by_mask);
+EXPORT_SYMBOL_GPL(dma_request_chan_by_domain);
 
 void dma_release_channel(struct dma_chan *chan)
 {
diff --git a/include/linux/dmaengine.h b/include/linux/dmaengine.h
index 8fcdee1c0cf9..de5c52810443 100644
--- a/include/linux/dmaengine.h
+++ b/include/linux/dmaengine.h
@@ -1307,7 +1307,8 @@ struct dma_chan *__dma_request_channel(const dma_cap_mask_t *mask,
 struct dma_chan *dma_request_slave_channel(struct device *dev, const char *name);
 
 struct dma_chan *dma_request_chan(struct device *dev, const char *name);
-struct dma_chan *dma_request_chan_by_mask(const dma_cap_mask_t *mask);
+struct dma_chan *dma_request_chan_by_domain(struct device *dev,
+					    const dma_cap_mask_t *mask);
 
 void dma_release_channel(struct dma_chan *chan);
 int dma_get_slave_caps(struct dma_chan *chan, struct dma_slave_caps *caps);
@@ -1344,8 +1345,8 @@ static inline struct dma_chan *dma_request_chan(struct device *dev,
 {
 	return ERR_PTR(-ENODEV);
 }
-static inline struct dma_chan *dma_request_chan_by_mask(
-						const dma_cap_mask_t *mask)
+static inline struct dma_chan *dma_request_chan_by_domain(struct device *dev,
+			const dma_cap_mask_t *mask)
 {
 	return ERR_PTR(-ENODEV);
 }
@@ -1406,6 +1407,8 @@ struct dma_chan *dma_get_any_slave_channel(struct dma_device *device);
 	__dma_request_channel(&(mask), x, y, NULL)
 #define dma_request_slave_channel_compat(mask, x, y, dev, name) \
 	__dma_request_slave_channel_compat(&(mask), x, y, dev, name)
+#define dma_request_chan_by_mask(mask) \
+	dma_request_chan_by_domain(NULL, mask)
 
 static inline struct dma_chan
 *__dma_request_slave_channel_compat(const dma_cap_mask_t *mask,
-- 
Peter

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains
  2019-09-10 11:50 ` [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains Peter Ujfalusi
@ 2019-09-13 14:36   ` Rob Herring
  2019-09-16 11:21     ` Peter Ujfalusi
  0 siblings, 1 reply; 6+ messages in thread
From: Rob Herring @ 2019-09-13 14:36 UTC (permalink / raw)
  To: Peter Ujfalusi; +Cc: vkoul, dmaengine, linux-kernel, dan.j.williams, devicetree

On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote:
> On systems where multiple DMA controllers available, non Slave (for example
> memcpy operation) users can not be described in DT as there is no device
> involved from the DMA controller's point of view, DMA binding is not usable.
> However in these systems still a peripheral might need to be serviced by or
> it is better to serviced by specific DMA controller.
> When a memcpy is used to/from a memory mapped region for example a DMA in the
> same domain can perform better.
> For generic software modules doing mem 2 mem operations it also matter that
> they will get a channel from a controller which is faster in DDR to DDR mode
> rather then from the first controller happen to be loaded.
> 
> This property is inherited, so it may be specified in a device node or in any
> of its parent nodes.

If a device needs mem2mem dma, I think we should just use the existing 
dma binding. The provider will need a way to define cell values which 
mean mem2mem.

For generic s/w, it should be able to query the dma speed or get a 
preferred one IMO. It's not a DT problem.

We measure memcpy speeds at boot time to select the fastest 
implementation for a chip, why not do that for mem2mem DMA?

> 
> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
> ---
>  .../devicetree/bindings/dma/dma-domain.yaml   | 88 +++++++++++++++++++
>  1 file changed, 88 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml

Note that you have several errors in your schema. Run 'make dt_bindings_check'.

Rob


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains
  2019-09-13 14:36   ` Rob Herring
@ 2019-09-16 11:21     ` Peter Ujfalusi
  0 siblings, 0 replies; 6+ messages in thread
From: Peter Ujfalusi @ 2019-09-16 11:21 UTC (permalink / raw)
  To: Rob Herring; +Cc: vkoul, dmaengine, linux-kernel, dan.j.williams, devicetree



On 13/09/2019 17.36, Rob Herring wrote:
> On Tue, Sep 10, 2019 at 02:50:35PM +0300, Peter Ujfalusi wrote:
>> On systems where multiple DMA controllers available, non Slave (for example
>> memcpy operation) users can not be described in DT as there is no device
>> involved from the DMA controller's point of view, DMA binding is not usable.
>> However in these systems still a peripheral might need to be serviced by or
>> it is better to serviced by specific DMA controller.
>> When a memcpy is used to/from a memory mapped region for example a DMA in the
>> same domain can perform better.
>> For generic software modules doing mem 2 mem operations it also matter that
>> they will get a channel from a controller which is faster in DDR to DDR mode
>> rather then from the first controller happen to be loaded.
>>
>> This property is inherited, so it may be specified in a device node or in any
>> of its parent nodes.
> 
> If a device needs mem2mem dma, I think we should just use the existing 
> dma binding. The provider will need a way to define cell values which 
> mean mem2mem.

But isn't it going to be an abuse of the binding? Each DMA controller
would hack this in different ways, probably using out of range DMA
request/trigger number or if they have direction in the binding or some
other parameter would be set to something invalid...

> For generic s/w, it should be able to query the dma speed or get a 
> preferred one IMO. It's not a DT problem.
> 
> We measure memcpy speeds at boot time to select the fastest 
> implementation for a chip, why not do that for mem2mem DMA?

It would make an impact on boot time since the tests would need to be
done with a large enough copy to be able to see clearly which one is faster.

Also we should be able to handle different probing orders:
client1 should have mem2mem channel from dma2.

- dma1 probes
- client1 probes and asks for a mem2mem channel
- dma2 probes

Here client1 should deffer until dma2 is probed.

Probably the property should be dma-mem2mem-domain to be more precise on
it's purpose and avoid confusion?

> 
>>
>> Signed-off-by: Peter Ujfalusi <peter.ujfalusi@ti.com>
>> ---
>>  .../devicetree/bindings/dma/dma-domain.yaml   | 88 +++++++++++++++++++
>>  1 file changed, 88 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/dma/dma-domain.yaml
> 
> Note that you have several errors in your schema. Run 'make dt_bindings_check'.

That does not do anything on my system, but git dt-doc-validate running
via https://github.com/robherring/yaml-bindings.git.

- Péter

Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, back to index

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-09-10 11:50 [PATCH 0/3] dmaengine: Support for DMA domain controllers Peter Ujfalusi
2019-09-10 11:50 ` [PATCH 1/3] dt-bindings: dma: Add documentation for DMA domains Peter Ujfalusi
2019-09-13 14:36   ` Rob Herring
2019-09-16 11:21     ` Peter Ujfalusi
2019-09-10 11:50 ` [PATCH 2/3] dmaengine: of_dma: Function to look up the DMA domain of a client Peter Ujfalusi
2019-09-10 11:50 ` [PATCH 3/3] dmaengine: Support for requesting channels preferring DMA domain controller Peter Ujfalusi

dmaengine Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dmaengine/0 dmaengine/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dmaengine dmaengine/ https://lore.kernel.org/dmaengine \
		dmaengine@vger.kernel.org dmaengine@archiver.kernel.org
	public-inbox-index dmaengine


Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dmaengine


AGPL code for this site: git clone https://public-inbox.org/ public-inbox