[PATCH 00/04] arm64: dts: r8a7795: Initial IPMMU upstream integration

[PATCH 00/04] arm64: dts: r8a7795: Initial IPMMU upstream integration

[Magnus Damm]

arm64: dts: r8a7795: Initial IPMMU upstream integration

[PATCH 01/04] arm64: dts: r8a7795: Add IPMMU device nodes
[PATCH 02/04] arm64: dts: r8a7795: Hook up SYS-DMAC to IPMMU
[PATCH 03/04] arm64: dts: r8a7795: Point FCP devices to IPMMU
[PATCH 04/04] arm64: dts: r8a7795: Connect Ethernet AVB to IPMMU

This series adds DT nodes for IPMMU devices on r8a7795 together
with connections to various other on-chip devices. Currently
all the IPMMU devices remain disabled, but once next hardware
iteration of r8a7795 becomes available (with errata fixes) and
various driver-level changes end up in mainline can start
enabling the IPMMU devices.

The DT binding for r8a7795 has since long been included in mainline
and this series implements support following such format:

d4e42e7 iommu/ipmmu-vmsa: Add r8a7795 DT binding

I hope these patches now can be queued up for upstream merge,
ideally aiming at v4.10-rc.

Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
---

 Developed on top of renesas-devel-20161024-v4.9-rc2

 No special run time or compile time dependencies.

 The first two patches earlier posted as
 [PATCH/RFC 00/05] arm64: dts: r8a7795: salvator-x: IPMMU test code V2

 arch/arm64/boot/dts/renesas/r8a7795.dtsi |  158 ++++++++++++++++++++++++++++++
 1 file changed, 158 insertions(+)

[PATCH 01/04] arm64: dts: r8a7795: Add IPMMU device nodes

[Magnus Damm]

From: Magnus Damm <damm+renesas@opensource.se>

Add r8a7795 IPMMU nodes and keep all disabled by default.

Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
---

 arch/arm64/boot/dts/renesas/r8a7795.dtsi |  113 ++++++++++++++++++++++++++++++
 1 file changed, 113 insertions(+)

--- 0001/arch/arm64/boot/dts/renesas/r8a7795.dtsi
+++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi	2016-10-27 18:03:38.940607110 +0900
@@ -413,6 +413,119 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 		};
 
+		ipmmu_vi: mmu@febd0000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xfebd0000 0 0x1000>; /* IPMMU-VI */
+			renesas,ipmmu-main = <&ipmmu_mm 11>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_vp: mmu@fe990000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xfe990000 0 0x1000>; /* IPMMU-VP */
+			renesas,ipmmu-main = <&ipmmu_mm 12>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_vc0: mmu@fe6b0000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xfe6b0000 0 0x1000>; /* IPMMU-VC0 */
+			renesas,ipmmu-main = <&ipmmu_mm 9>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_vc1: mmu@fe6f0000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xfe6f0000 0 0x1000>; /* IPMMU-VC1 */
+			renesas,ipmmu-main = <&ipmmu_mm 10>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_pv0: mmu@fd800000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xfd800000 0 0x1000>; /* IPMMU-PV0 */
+			renesas,ipmmu-main = <&ipmmu_mm 6>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_ir: mmu@ff8b0000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xff8b0000 0 0x1000>; /* IPMMU-IR */
+			renesas,ipmmu-main = <&ipmmu_mm 3>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_hc: mmu@e6570000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xe6570000 0 0x1000>; /* IPMMU-HC */
+			renesas,ipmmu-main = <&ipmmu_mm 2>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_rt: mmu@ffc80000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xffc80000 0 0x1000>; /* IPMMU-RT */
+			renesas,ipmmu-main = <&ipmmu_mm 7>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_mp0: mmu@ec670000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xec670000 0 0x1000>; /* IPMMU-MP0 */
+			renesas,ipmmu-main = <&ipmmu_mm 4>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_mp1: mmu@ec680000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xec680000 0 0x1000>; /* IPMMU-MP1 */
+			renesas,ipmmu-main = <&ipmmu_mm 5>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_sy: mmu@e7730000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xe7730000 0 0x1000>; /* IPMMU-SY */
+			renesas,ipmmu-main = <&ipmmu_mm 8>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_ds0: mmu@e6740000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xe6740000 0 0x1000>; /* IPMMU-DS0 */
+			renesas,ipmmu-main = <&ipmmu_mm 0>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_ds1: mmu@e7740000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xe7740000 0 0x1000>; /* IPMMU-DS1 */
+			renesas,ipmmu-main = <&ipmmu_mm 1>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
+		ipmmu_mm: mmu@e67b0000 {
+			compatible = "renesas,ipmmu-r8a7795";
+			reg = <0 0xe67b0000 0 0x1000>; /* IPMMU-MM */
+			interrupts = <GIC_SPI 196 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 197 IRQ_TYPE_LEVEL_HIGH>;
+			#iommu-cells = <1>;
+			status = "disabled";
+		};
+
 		dmac0: dma-controller@e6700000 {
 			compatible = "renesas,dmac-r8a7795",
 				     "renesas,rcar-dmac";

[PATCH 02/04] arm64: dts: r8a7795: Hook up SYS-DMAC to IPMMU

[Magnus Damm]

From: Magnus Damm <damm+renesas@opensource.se>

Hook up r8a7795 DMAC nodes to IPMMU-MP1, IPMMU-DS0 and IPMMU-DS1.

Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
---

 This patch can be merged any time, but it is however not recommended
 to enable IPMMU-MP1, IPMMU-DS0 or IPMMU-DS1 until various dependencies
 have been resolved:

 1) DMA-Engine slave devices need the following two patches merged:
  arm64: Wire up iommu_dma_{map, unmap}_resource()
  iommu/dma: Implement dma_{map,unmap}_resource()

 2) The Audio DMAC and the sound drivers need the above two patches
  as well as driver updates to correctly make use of the DMA MAP API.

 arch/arm64/boot/dts/renesas/r8a7795.dtsi |   40 ++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)

--- 0002/arch/arm64/boot/dts/renesas/r8a7795.dtsi
+++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi	2016-10-27 18:04:48.460607110 +0900
@@ -358,6 +358,14 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 			#dma-cells = <1>;
 			dma-channels = <16>;
+			iommus = <&ipmmu_mp1 0>, <&ipmmu_mp1 1>,
+			       <&ipmmu_mp1 2>, <&ipmmu_mp1 3>,
+			       <&ipmmu_mp1 4>, <&ipmmu_mp1 5>,
+			       <&ipmmu_mp1 6>, <&ipmmu_mp1 7>,
+			       <&ipmmu_mp1 8>, <&ipmmu_mp1 9>,
+			       <&ipmmu_mp1 10>, <&ipmmu_mp1 11>,
+			       <&ipmmu_mp1 12>, <&ipmmu_mp1 13>,
+			       <&ipmmu_mp1 14>, <&ipmmu_mp1 15>;
 		};
 
 		audma1: dma-controller@ec720000 {
@@ -391,6 +399,14 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 			#dma-cells = <1>;
 			dma-channels = <16>;
+			iommus = <&ipmmu_mp1 16>, <&ipmmu_mp1 17>,
+			       <&ipmmu_mp1 18>, <&ipmmu_mp1 19>,
+			       <&ipmmu_mp1 20>, <&ipmmu_mp1 21>,
+			       <&ipmmu_mp1 22>, <&ipmmu_mp1 23>,
+			       <&ipmmu_mp1 24>, <&ipmmu_mp1 25>,
+			       <&ipmmu_mp1 26>, <&ipmmu_mp1 27>,
+			       <&ipmmu_mp1 28>, <&ipmmu_mp1 29>,
+			       <&ipmmu_mp1 30>, <&ipmmu_mp1 31>;
 		};
 
 		pfc: pfc@e6060000 {
@@ -557,6 +573,14 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 			#dma-cells = <1>;
 			dma-channels = <16>;
+			iommus = <&ipmmu_ds0 0>, <&ipmmu_ds0 1>,
+			       <&ipmmu_ds0 2>, <&ipmmu_ds0 3>,
+			       <&ipmmu_ds0 4>, <&ipmmu_ds0 5>,
+			       <&ipmmu_ds0 6>, <&ipmmu_ds0 7>,
+			       <&ipmmu_ds0 8>, <&ipmmu_ds0 9>,
+			       <&ipmmu_ds0 10>, <&ipmmu_ds0 11>,
+			       <&ipmmu_ds0 12>, <&ipmmu_ds0 13>,
+			       <&ipmmu_ds0 14>, <&ipmmu_ds0 15>;
 		};
 
 		dmac1: dma-controller@e7300000 {
@@ -590,6 +614,14 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 			#dma-cells = <1>;
 			dma-channels = <16>;
+			iommus = <&ipmmu_ds1 0>, <&ipmmu_ds1 1>,
+			       <&ipmmu_ds1 2>, <&ipmmu_ds1 3>,
+			       <&ipmmu_ds1 4>, <&ipmmu_ds1 5>,
+			       <&ipmmu_ds1 6>, <&ipmmu_ds1 7>,
+			       <&ipmmu_ds1 8>, <&ipmmu_ds1 9>,
+			       <&ipmmu_ds1 10>, <&ipmmu_ds1 11>,
+			       <&ipmmu_ds1 12>, <&ipmmu_ds1 13>,
+			       <&ipmmu_ds1 14>, <&ipmmu_ds1 15>;
 		};
 
 		dmac2: dma-controller@e7310000 {
@@ -623,6 +655,14 @@
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
 			#dma-cells = <1>;
 			dma-channels = <16>;
+			iommus = <&ipmmu_ds1 16>, <&ipmmu_ds1 17>,
+			       <&ipmmu_ds1 18>, <&ipmmu_ds1 19>,
+			       <&ipmmu_ds1 20>, <&ipmmu_ds1 21>,
+			       <&ipmmu_ds1 22>, <&ipmmu_ds1 23>,
+			       <&ipmmu_ds1 24>, <&ipmmu_ds1 25>,
+			       <&ipmmu_ds1 26>, <&ipmmu_ds1 27>,
+			       <&ipmmu_ds1 28>, <&ipmmu_ds1 29>,
+			       <&ipmmu_ds1 30>, <&ipmmu_ds1 31>;
 		};
 
 		avb: ethernet@e6800000 {

[PATCH 03/04] arm64: dts: r8a7795: Point FCP devices to IPMMU

[Magnus Damm]

From: Magnus Damm <damm+renesas@opensource.se>

Hook in the FCP devices to allow use of the VSP and DU
together with the IPMMU.

Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
---

 The patch can be merged as is, but it is recommended not
 to enable IPMMU-VI until the DU driver gets updated in
 mainline with the following patch series and further
 updates:
 
 [PATCH 0/6] R-Car DU: Fix IOMMU operation when connected to VSP
 
 arch/arm64/boot/dts/renesas/r8a7795.dtsi |    4 ++++
 1 file changed, 4 insertions(+)

--- 0004/arch/arm64/boot/dts/renesas/r8a7795.dtsi
+++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi	2016-10-27 18:06:14.180607110 +0900
@@ -1569,6 +1569,7 @@
 			reg = <0 0xfea27000 0 0x200>;
 			clocks = <&cpg CPG_MOD 603>;
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+			iommus = <&ipmmu_vi 8>;
 		};
 
 		vspd1: vsp@fea28000 {
@@ -1586,6 +1587,7 @@
 			reg = <0 0xfea2f000 0 0x200>;
 			clocks = <&cpg CPG_MOD 602>;
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+			iommus = <&ipmmu_vi 9>;
 		};
 
 		vspd2: vsp@fea30000 {
@@ -1603,6 +1605,7 @@
 			reg = <0 0xfea37000 0 0x200>;
 			clocks = <&cpg CPG_MOD 601>;
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+			iommus = <&ipmmu_vi 10>;
 		};
 
 		vspd3: vsp@fea38000 {
@@ -1620,6 +1623,7 @@
 			reg = <0 0xfea3f000 0 0x200>;
 			clocks = <&cpg CPG_MOD 600>;
 			power-domains = <&sysc R8A7795_PD_ALWAYS_ON>;
+			iommus = <&ipmmu_vi 11>;
 		};
 
 		fdp1@fe940000 {

[PATCH 04/04] arm64: dts: r8a7795: Connect Ethernet AVB to IPMMU

[Magnus Damm]

From: Magnus Damm <damm+renesas@opensource.se>

Add IPMMU-DS0 to the Ethernet-AVB device node.

Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
---

 arch/arm64/boot/dts/renesas/r8a7795.dtsi |    1 +
 1 file changed, 1 insertion(+)

--- 0006/arch/arm64/boot/dts/renesas/r8a7795.dtsi
+++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi	2016-10-27 18:08:09.590607110 +0900
@@ -706,6 +706,7 @@
 			phy-mode = "rgmii-id";
 			#address-cells = <1>;
 			#size-cells = <0>;
+			iommus = <&ipmmu_ds0 16>;
 		};
 
 		can0: can@e6c30000 {

Re: [PATCH 02/04] arm64: dts: r8a7795: Hook up SYS-DMAC to IPMMU

[Geert Uytterhoeven]

Hi Magnus,

On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
> From: Magnus Damm <damm+renesas@opensource.se>
>
> Hook up r8a7795 DMAC nodes to IPMMU-MP1, IPMMU-DS0 and IPMMU-DS1.
>
> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
> ---
>
>  This patch can be merged any time, but it is however not recommended
>  to enable IPMMU-MP1, IPMMU-DS0 or IPMMU-DS1 until various dependencies
>  have been resolved:

Forgive my ignorance, but how does the driver core treat devices with
iommus properties pointing to disabled IOMMU nodes?
Is this ignored silently, or does this cause -EPROBEDEFER, like for clocks
and power-domains?

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 02/04] arm64: dts: r8a7795: Hook up SYS-DMAC to IPMMU

[Magnus Damm]

Hi Geert,

On Thu, Oct 27, 2016 at 7:42 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> Hi Magnus,
>
> On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
>> From: Magnus Damm <damm+renesas@opensource.se>
>>
>> Hook up r8a7795 DMAC nodes to IPMMU-MP1, IPMMU-DS0 and IPMMU-DS1.
>>
>> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
>> ---
>>
>>  This patch can be merged any time, but it is however not recommended
>>  to enable IPMMU-MP1, IPMMU-DS0 or IPMMU-DS1 until various dependencies
>>  have been resolved:
>
> Forgive my ignorance, but how does the driver core treat devices with
> iommus properties pointing to disabled IOMMU nodes?
> Is this ignored silently, or does this cause -EPROBEDEFER, like for clocks
> and power-domains?

Not sure about current state of the driver core to be honest. Earlier
I needed to add a local workaround in the ->xlate() callback in the
IPMMU driver but I need to revisit to see if that needs to be updated.
Any ideas?

I do however know that the IPMMU driver stack included in
renesas-drivers works both with and without iommus properties and in
case an iommus property is used then both enabled and disabled are
known to work.

Looking at mainline, at this point the IPMMU driver changes for
r8a7795 and r8a7796 are not included yet, so there is no driver code
that will match with the DT compat string. Once we queue up the IPMMU
driver code for merge then we need to make sure it still works as
expected.

I just booted r8a7795 Salvator-X with these patches on top of
renesas-devel-20161024-v4.9-rc2 (that lacks the IPMMU driver for
r8a7795) and the DU device that is connected via FCP and VSP operate
as usual. So all seems fine what I can tell.

Thanks,

/ magnus

Re: [PATCH 02/04] arm64: dts: r8a7795: Hook up SYS-DMAC to IPMMU

[Geert Uytterhoeven]

Hi Magnus,

On Thu, Oct 27, 2016 at 1:07 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
> On Thu, Oct 27, 2016 at 7:42 PM, Geert Uytterhoeven
> <geert@linux-m68k.org> wrote:
>> On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
>>> From: Magnus Damm <damm+renesas@opensource.se>
>>>
>>> Hook up r8a7795 DMAC nodes to IPMMU-MP1, IPMMU-DS0 and IPMMU-DS1.
>>>
>>> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
>>> ---
>>>
>>>  This patch can be merged any time, but it is however not recommended
>>>  to enable IPMMU-MP1, IPMMU-DS0 or IPMMU-DS1 until various dependencies
>>>  have been resolved:
>>
>> Forgive my ignorance, but how does the driver core treat devices with
>> iommus properties pointing to disabled IOMMU nodes?
>> Is this ignored silently, or does this cause -EPROBEDEFER, like for clocks
>> and power-domains?
>
> Not sure about current state of the driver core to be honest. Earlier
> I needed to add a local workaround in the ->xlate() callback in the
> IPMMU driver but I need to revisit to see if that needs to be updated.
> Any ideas?
>
> I do however know that the IPMMU driver stack included in
> renesas-drivers works both with and without iommus properties and in
> case an iommus property is used then both enabled and disabled are
> known to work.
>
> Looking at mainline, at this point the IPMMU driver changes for
> r8a7795 and r8a7796 are not included yet, so there is no driver code
> that will match with the DT compat string. Once we queue up the IPMMU
> driver code for merge then we need to make sure it still works as
> expected.

Not finding a driver for a compat string and a disabled node should behave
the same (although we had issues in the past where the disabled was ignored).

> I just booted r8a7795 Salvator-X with these patches on top of
> renesas-devel-20161024-v4.9-rc2 (that lacks the IPMMU driver for
> r8a7795) and the DU device that is connected via FCP and VSP operate
> as usual. So all seems fine what I can tell.

Thanks for checking!

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 04/04] arm64: dts: r8a7795: Connect Ethernet AVB to IPMMU

[Geert Uytterhoeven]

Hi Magnus,

On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
> From: Magnus Damm <damm+renesas@opensource.se>
>
> Add IPMMU-DS0 to the Ethernet-AVB device node.
>
> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>

Thanks for your patch!

> ---
>
>  arch/arm64/boot/dts/renesas/r8a7795.dtsi |    1 +
>  1 file changed, 1 insertion(+)
>
> --- 0006/arch/arm64/boot/dts/renesas/r8a7795.dtsi
> +++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi       2016-10-27 18:08:09.590607110 +0900
> @@ -706,6 +706,7 @@
>                         phy-mode = "rgmii-id";
>                         #address-cells = <1>;
>                         #size-cells = <0>;
> +                       iommus = <&ipmmu_ds0 16>;
>                 };

I've applied the same to r8a7796.dtsi, and enabled the ipmmu_ds0 and
ipmmu_mm nodes.

With CONFIG_DMA_API_DEBUG=y, I get:

ipmmu-vmsa e67b0000.mmu: DMA-API: device driver tries to sync DMA
memory it has not allocated [device address=0x000000067b9d2018]
[size=8 bytes]
------------[ cut here ]------------
WARNING: CPU: 0 PID: 1 at lib/dma-debug.c:1234 check_sync+0xcc/0x568
Modules linked in:

CPU: 0 PID: 1 Comm: swapper/0 Not tainted
4.9.0-rc2-salvator-x-01968-g5193fb91213b0165-dirty #61
Hardware name: Renesas Salvator-X board based on r8a7796 (DT)
task: ffffffc63b860080 task.stack: ffffffc63b864000
PC is at check_sync+0xcc/0x568
LR is at check_sync+0xcc/0x568
pc : [<ffffff8008234a58>] lr : [<ffffff8008234a58>] pstate: 600000c5
sp : ffffffc63b8677f0
x29: ffffffc63b8677f0 x28: ffffffc63b98f910
x27: 0000000000000000 x26: 0000000000001000
x25: ffffffc63b9d2018 x24: 0000000000000001
x23: 00000000fffff000 x22: 0000000000000000
x21: ffffff8008c05000 x20: ffffffc63b867850
x19: ffffffc63b991810 x18: 00000000e5b7a57f
x17: 00000000a2deee83 x16: 000000003ed9b3b5
x15: 00000000578b233a x14: 796220383d657a69
x13: 735b205d38313032 x12: 6439623736303030
x11: 3030303078303d73 x10: 7365726464612065
x9 : 63697665645b2064 x8 : 657461636f6c6c61
x7 : 20746f6e20736168 x6 : ffffff8008c6a5b7
x5 : 0000000000000022 x4 : 0000000000000010
x3 : 0000000000000000 x2 : 00000000dead4ead
x1 : ffffff8008c19370 x0 : 0000000000000090

---[ end trace 9e4d7153ac803e17 ]---
Call trace:
Exception stack(0xffffffc63b867620 to 0xffffffc63b867750)
7620: ffffffc63b991810 0000008000000000 ffffffc63b8677f0 ffffff8008234a58
7640: ffffff8008c05838 ffffff8008c058e0 ffffff800877cb09 00000007080d869c
7660: 00000000000001c7 0000000100000000 ffffffc63b867710 ffffff80080d8884
7680: ffffffc63b991810 ffffffc63b867850 ffffff8008c05000 0000000000000000
76a0: 00000000fffff000 0000000000000001 ffffffc63b9d2018 0000000000001000
76c0: 0000000000000090 ffffff8008c19370 00000000dead4ead 0000000000000000
76e0: 0000000000000010 0000000000000022 ffffff8008c6a5b7 20746f6e20736168
7700: 657461636f6c6c61 63697665645b2064 7365726464612065 3030303078303d73
7720: 6439623736303030 735b205d38313032 796220383d657a69 00000000578b233a
7740: 000000003ed9b3b5 00000000a2deee83
[<ffffff8008234a58>] check_sync+0xcc/0x568
[<ffffff8008234f84>] debug_dma_sync_single_for_device+0x44/0x4c
[<ffffff80082b1968>] __arm_lpae_set_pte.isra.3+0x8c/0x98
[<ffffff80082b1bf4>] __arm_lpae_map+0x280/0x2dc
[<ffffff80082b211c>] arm_lpae_map+0xb0/0xc4
[<ffffff80082b368c>] ipmmu_map+0x20/0x30
[<ffffff80082aeff8>] iommu_map+0xd4/0x128
[<ffffff80082af13c>] default_iommu_map_sg+0xf0/0x148
[<ffffff80082b067c>] iommu_dma_alloc+0x300/0x344
[<ffffff8008092e4c>] __iommu_alloc_attrs+0xb4/0x1f4
[<ffffff8008359794>] ravb_probe+0x474/0x820
[<ffffff80082ee7d8>] platform_drv_probe+0x58/0xa4
[<ffffff80082ece44>] driver_probe_device+0x124/0x278
[<ffffff80082ed00c>] __driver_attach+0x74/0xa0
[<ffffff80082eb380>] bus_for_each_dev+0x64/0x88
[<ffffff80082ec910>] driver_attach+0x20/0x28
[<ffffff80082ec4d8>] bus_add_driver+0xe4/0x1e8
[<ffffff80082ed8a4>] driver_register+0x98/0xe4
[<ffffff80082ee730>] __platform_driver_register+0x48/0x50
[<ffffff8008a216dc>] ravb_driver_init+0x18/0x20
[<ffffff8008083190>] do_one_initcall+0x88/0x108
[<ffffff8008a00c9c>] kernel_init_freeable+0x148/0x1e4
[<ffffff8008502284>] kernel_init+0x10/0xfc
[<ffffff8008082e80>] ret_from_fork+0x10/0x50

Gives this happens inside the IPMMU driver, called from the first
dma_alloc_coherent() in ravb_probe(), I think this is a bug in the IPMMU
driver, not in the RAVB driver.

Nevertheless, RAVB itself seems to work fine with IPMMU enabled.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 04/04] arm64: dts: r8a7795: Connect Ethernet AVB to IPMMU

[Geert Uytterhoeven]

Hi Magnus,

On Fri, Oct 28, 2016 at 2:40 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
>> From: Magnus Damm <damm+renesas@opensource.se>
>>
>> Add IPMMU-DS0 to the Ethernet-AVB device node.
>>
>> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
>
> Thanks for your patch!
>
>> ---
>>
>>  arch/arm64/boot/dts/renesas/r8a7795.dtsi |    1 +
>>  1 file changed, 1 insertion(+)
>>
>> --- 0006/arch/arm64/boot/dts/renesas/r8a7795.dtsi
>> +++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi       2016-10-27 18:08:09.590607110 +0900
>> @@ -706,6 +706,7 @@
>>                         phy-mode = "rgmii-id";
>>                         #address-cells = <1>;
>>                         #size-cells = <0>;
>> +                       iommus = <&ipmmu_ds0 16>;
>>                 };
>
> I've applied the same to r8a7796.dtsi, and enabled the ipmmu_ds0 and
> ipmmu_mm nodes.
>
> With CONFIG_DMA_API_DEBUG=y, I get:
>
> ipmmu-vmsa e67b0000.mmu: DMA-API: device driver tries to sync DMA
> memory it has not allocated [device address=0x000000067b9d2018]
> [size=8 bytes]
> ------------[ cut here ]------------
> WARNING: CPU: 0 PID: 1 at lib/dma-debug.c:1234 check_sync+0xcc/0x568
> Modules linked in:
>
> CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 4.9.0-rc2-salvator-x-01968-g5193fb91213b0165-dirty #61
> Hardware name: Renesas Salvator-X board based on r8a7796 (DT)
> task: ffffffc63b860080 task.stack: ffffffc63b864000
> PC is at check_sync+0xcc/0x568
> LR is at check_sync+0xcc/0x568
> pc : [<ffffff8008234a58>] lr : [<ffffff8008234a58>] pstate: 600000c5
> sp : ffffffc63b8677f0
> x29: ffffffc63b8677f0 x28: ffffffc63b98f910
> x27: 0000000000000000 x26: 0000000000001000
> x25: ffffffc63b9d2018 x24: 0000000000000001
> x23: 00000000fffff000 x22: 0000000000000000
> x21: ffffff8008c05000 x20: ffffffc63b867850
> x19: ffffffc63b991810 x18: 00000000e5b7a57f
> x17: 00000000a2deee83 x16: 000000003ed9b3b5
> x15: 00000000578b233a x14: 796220383d657a69
> x13: 735b205d38313032 x12: 6439623736303030
> x11: 3030303078303d73 x10: 7365726464612065
> x9 : 63697665645b2064 x8 : 657461636f6c6c61
> x7 : 20746f6e20736168 x6 : ffffff8008c6a5b7
> x5 : 0000000000000022 x4 : 0000000000000010
> x3 : 0000000000000000 x2 : 00000000dead4ead
> x1 : ffffff8008c19370 x0 : 0000000000000090
>
> ---[ end trace 9e4d7153ac803e17 ]---
> Call trace:
> Exception stack(0xffffffc63b867620 to 0xffffffc63b867750)
> 7620: ffffffc63b991810 0000008000000000 ffffffc63b8677f0 ffffff8008234a58
> 7640: ffffff8008c05838 ffffff8008c058e0 ffffff800877cb09 00000007080d869c
> 7660: 00000000000001c7 0000000100000000 ffffffc63b867710 ffffff80080d8884
> 7680: ffffffc63b991810 ffffffc63b867850 ffffff8008c05000 0000000000000000
> 76a0: 00000000fffff000 0000000000000001 ffffffc63b9d2018 0000000000001000
> 76c0: 0000000000000090 ffffff8008c19370 00000000dead4ead 0000000000000000
> 76e0: 0000000000000010 0000000000000022 ffffff8008c6a5b7 20746f6e20736168
> 7700: 657461636f6c6c61 63697665645b2064 7365726464612065 3030303078303d73
> 7720: 6439623736303030 735b205d38313032 796220383d657a69 00000000578b233a
> 7740: 000000003ed9b3b5 00000000a2deee83
> [<ffffff8008234a58>] check_sync+0xcc/0x568
> [<ffffff8008234f84>] debug_dma_sync_single_for_device+0x44/0x4c
> [<ffffff80082b1968>] __arm_lpae_set_pte.isra.3+0x8c/0x98
> [<ffffff80082b1bf4>] __arm_lpae_map+0x280/0x2dc
> [<ffffff80082b211c>] arm_lpae_map+0xb0/0xc4
> [<ffffff80082b368c>] ipmmu_map+0x20/0x30

I found another issue when booting with swiotlb=force.
In that case, ipmmu_map() crashes with a NULL pointer deference, as
domain->iop is NULL due to a failed call to alloc_io_pgtable_ops() in
ipmmu_domain_init_context() before.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: [PATCH 04/04] arm64: dts: r8a7795: Connect Ethernet AVB to IPMMU

[Geert Uytterhoeven]

Hi Magnus,

On Thu, Oct 27, 2016 at 12:29 PM, Magnus Damm <magnus.damm@gmail.com> wrote:
> From: Magnus Damm <damm+renesas@opensource.se>
>
> Add IPMMU-DS0 to the Ethernet-AVB device node.
>
> Signed-off-by: Magnus Damm <damm+renesas@opensource.se>
> ---
>
>  arch/arm64/boot/dts/renesas/r8a7795.dtsi |    1 +
>  1 file changed, 1 insertion(+)
>
> --- 0006/arch/arm64/boot/dts/renesas/r8a7795.dtsi
> +++ work/arch/arm64/boot/dts/renesas/r8a7795.dtsi       2016-10-27 18:08:09.590607110 +0900
> @@ -706,6 +706,7 @@
>                         phy-mode = "rgmii-id";
>                         #address-cells = <1>;
>                         #size-cells = <0>;
> +                       iommus = <&ipmmu_ds0 16>;

For consistency, you may want to add this line just below the power-domains
property.

>                 };

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds