[PATCH 00/12] i.MX media devices and connections

[PATCH 00/12] i.MX media devices and connections

[Steve Longerbeam]

Hi Philipp, Sascha, Shawn, et al,

I've been working for the past few months on a media driver for i.MX.
In addition to the media entities for the IPU-external units involved
with video capture (video mux and MIPI CSI-2 receiver), I've created
media entities for the IPU CSI, SMFC, and IC subunits. The IC entities
carry out scaling, CSC, horizontal/vertical flip, and rotation. In
addition, the IC-PRPVF entity carries out motion compensated
de-interlace.

The following series adds the OF device nodes and graphs that define
all the possible hardware connections supported by the i.MX involved
in video capture and image conversion.

Here are some of the pipelines defined by the OF graphs:

CSI -> IC-PRPENC
CSI -> IC-PRPVF
CSI -> IC-PRPVF -> IC-PP
CSI -> SMFC
CSI -> SMFC -> IC-PRPVF
CSI -> SMFC -> IC-PP
CSI -> SMFC -> IC-PRPVF -> IC-PP

You will notice that three IC-PP nodes are defined (ipu1_ic_pp0,
ipu1_ic_pp1, ipu1_ic_pp2, and same for ipu2). The reason for that
is that the IC-PP media entity uses the new ipu-image-conversion
API, which allows for multiple conversion contexts to be created.
Each IC-PP entity thus creates its own conversion context, and there
can be any number of IC-PP entities instantiated as needed by the OF
graph.

Camera sensor nodes are also added for the SabreAuto, SabreSD, and
SabreLite reference platforms.

The media driver is now in fairly good shape. It parses the OF graphs
to create the media pads and links. All the pipelines defined by the
OF graphs have been tested and are working. My media driver work is
at:

git@github.com:slongerbeam/mediatree.git, branch imx-media-staging-md-v2.

For an overview of the pipelines supported and usage notes for the
reference boards, you can refer to Documentation/media/v4l-drivers/imx.rst.

I realize there is collision here with the recent patch series posted by
Philipp, particularly around the video multiplexer and mipi csi-2 receiver
subdevs and OF graphs, as well as v4l2 capture drivers.



Philipp Zabel (1):
  ARM: dts: imx6qdl: add video capture devices and connections

Steve Longerbeam (11):
  ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node
  ARM: dts: imx6qdl: rename ipu client nodes
  ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors
  ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors
  ARM: dts: imx6-sabreauto: create i2cmux for i2c3
  ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b
  ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture
  ARM: dts: imx6-sabreauto: add the ADV7180 video decoder
  gpu: ipu-v3: Add ipu_unit_type enumeration
  gpu: ipu-v3: lookup ipu client nodes by name
  gpu: ipu-v3: Add smfc and ic client devices

 arch/arm/boot/dts/imx6dl-sabrelite.dts   |   5 +
 arch/arm/boot/dts/imx6dl-sabresd.dts     |   5 +
 arch/arm/boot/dts/imx6dl.dtsi            | 190 ++++++++++++
 arch/arm/boot/dts/imx6q-sabrelite.dts    |   6 +
 arch/arm/boot/dts/imx6q-sabresd.dts      |   5 +
 arch/arm/boot/dts/imx6q.dtsi             | 497 ++++++++++++++++++++++++++++++-
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 148 +++++++--
 arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 122 +++++++-
 arch/arm/boot/dts/imx6qdl-sabresd.dtsi   | 114 ++++++-
 arch/arm/boot/dts/imx6qdl.dtsi           | 385 +++++++++++++++++++++++-
 drivers/gpu/ipu-v3/ipu-common.c          | 142 ++++++++-
 include/video/imx-ipu-v3.h               |  21 ++
 12 files changed, 1593 insertions(+), 47 deletions(-)

-- 
2.7.4

[PATCH 01/12] ARM: dts: imx6qdl: Add compatible, clocks, irqs to MIPI CSI-2 node

[Steve Longerbeam]

Add to the MIPI CSI2 receiver node: compatible string, interrupt sources,
clocks.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl.dtsi | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index b13b0b2..e01e5d5 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1121,7 +1121,14 @@
 			};
 
 			mipi_csi: mipi@021dc000 {
+				compatible = "fsl,imx-mipi-csi2";
 				reg = <0x021dc000 0x4000>;
+				interrupts = <0 100 0x04>, <0 101 0x04>;
+				clocks = <&clks IMX6QDL_CLK_HSI_TX>,
+					 <&clks IMX6QDL_CLK_VIDEO_27M>,
+					 <&clks IMX6QDL_CLK_EIM_SEL>;
+				clock-names = "dphy_clk", "cfg_clk", "pix_clk";
+				status = "disabled";
 			};
 
 			mipi_dsi: mipi@021e0000 {
-- 
2.7.4

[PATCH 02/12] ARM: dts: imx6qdl: rename ipu client nodes

[Steve Longerbeam]

To allow for IPU client devices that are composed of more than one
port for input and output (SMFC and IC), change the nodes from being
a single port node to nodes that can contain multiple ports. Rename
the nodes to use the following format: "ipu<id>_<subunit>".

The IPUv3 driver will then need to lookup the client nodes by name
rather than by port id.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6q.dtsi   | 12 ++++++------
 arch/arm/boot/dts/imx6qdl.dtsi | 12 ++++++------
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index e9a5d0b..2b261ba 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -141,18 +141,18 @@
 			clock-names = "bus", "di0", "di1";
 			resets = <&src 4>;
 
-			ipu2_csi0: port@0 {
+			ipu2_csi0: ipu2_csi@0 {
 				reg = <0>;
 			};
 
-			ipu2_csi1: port@1 {
+			ipu2_csi1: ipu2_csi@1 {
 				reg = <1>;
 			};
 
-			ipu2_di0: port@2 {
+			ipu2_di0: ipu2_di@0 {
 				#address-cells = <1>;
 				#size-cells = <0>;
-				reg = <2>;
+				reg = <0>;
 
 				ipu2_di0_disp0: disp0-endpoint {
 				};
@@ -174,10 +174,10 @@
 				};
 			};
 
-			ipu2_di1: port@3 {
+			ipu2_di1: ipu2_di@1 {
 				#address-cells = <1>;
 				#size-cells = <0>;
-				reg = <3>;
+				reg = <1>;
 
 				ipu2_di1_hdmi: hdmi-endpoint {
 					remote-endpoint = <&hdmi_mux_3>;
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index e01e5d5..2465187 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1226,18 +1226,18 @@
 			clock-names = "bus", "di0", "di1";
 			resets = <&src 2>;
 
-			ipu1_csi0: port@0 {
+			ipu1_csi0: ipu1_csi@0 {
 				reg = <0>;
 			};
 
-			ipu1_csi1: port@1 {
+			ipu1_csi1: ipu1_csi@1 {
 				reg = <1>;
 			};
 
-			ipu1_di0: port@2 {
+			ipu1_di0: ipu1_di@0 {
 				#address-cells = <1>;
 				#size-cells = <0>;
-				reg = <2>;
+				reg = <0>;
 
 				ipu1_di0_disp0: disp0-endpoint {
 				};
@@ -1259,10 +1259,10 @@
 				};
 			};
 
-			ipu1_di1: port@3 {
+			ipu1_di1: ipu1_di@1 {
 				#address-cells = <1>;
 				#size-cells = <0>;
-				reg = <3>;
+				reg = <1>;
 
 				ipu1_di1_disp1: disp1-endpoint {
 				};
-- 
2.7.4

[PATCH 03/12] ARM: dts: imx6qdl: add video capture devices and connections

[Steve Longerbeam]

From: Philipp Zabel <p.zabel@pengutronix.de>

This patch adds the IPU subunit devices involved in video capture and
image conversion, and defines all the possible hardware connections
between them via OF graphs.

External to the IPU:

Video input multiplexers are defined that multiplex inputs from camera
sensors and the MIPI-CSI2 gasket, to the IPU CSIs.

On i.MX6Q/D two two-input multiplexers in front of IPU1 CSI0 and IPU2 CSI1
allow to select between CSI0/1 parallel input pads and the MIPI CSI-2 virtual
channels 0/3.

On i.MX6DL/S two five-input multiplexers in front of IPU1 CSI0 and IPU1 CSI1
allow to select between CSI0/1 parallel input pads and any of the four MIPI
CSI-2 virtual channels.

Internal to the IPU:

The IPU CSI, SMFC, IC-PRPENC, IC-PRPVF, and IC-PP subunits are added
as children of the IPUs, along with the hardware-supported connections
between them.

Finally, a media device node is defined. A video camera interface
and mem2mem device are defined as children of the media device.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
Signed-off-by: Philipp Zabel <p.zabel@pengutronix.de>
---
 arch/arm/boot/dts/imx6dl.dtsi  | 190 ++++++++++++++++
 arch/arm/boot/dts/imx6q.dtsi   | 487 +++++++++++++++++++++++++++++++++++++++++
 arch/arm/boot/dts/imx6qdl.dtsi | 368 +++++++++++++++++++++++++++++++
 3 files changed, 1045 insertions(+)

diff --git a/arch/arm/boot/dts/imx6dl.dtsi b/arch/arm/boot/dts/imx6dl.dtsi
index 1ade195..4bab076 100644
--- a/arch/arm/boot/dts/imx6dl.dtsi
+++ b/arch/arm/boot/dts/imx6dl.dtsi
@@ -109,6 +109,120 @@
 		compatible = "fsl,imx-gpu-subsystem";
 		cores = <&gpu_2d>, <&gpu_3d>;
 	};
+
+	ipu1_csi0_mux: ipu1_csi0_mux@34 {
+		compatible = "imx-video-mux";
+		reg = <0x34 0x07>;
+		gpr = <&gpr>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <5>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu1_csi0_mux_from_mipi_vc0: endpoint {
+				remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu1_csi0_mux_from_mipi_vc1: endpoint {
+				remote-endpoint = <&mipi_vc1_to_ipu1_csi0_mux>;
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu1_csi0_mux_from_mipi_vc2: endpoint {
+				remote-endpoint = <&mipi_vc2_to_ipu1_csi0_mux>;
+			};
+		};
+
+		port@3 {
+			reg = <3>;
+
+			ipu1_csi0_mux_from_mipi_vc3: endpoint {
+				remote-endpoint = <&mipi_vc3_to_ipu1_csi0_mux>;
+			};
+		};
+
+		port@4 {
+			reg = <4>;
+
+			ipu1_csi0_mux_from_parallel_sensor: endpoint {
+			};
+		};
+
+		port@5 {
+			reg = <5>;
+
+			ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+				remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+			};
+		};
+	};
+
+	ipu1_csi1_mux: ipu1_csi1_mux@34 {
+		compatible = "imx-video-mux";
+		reg = <0x34 0x38>;
+		gpr = <&gpr>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <5>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu1_csi1_mux_from_mipi_vc0: endpoint {
+				remote-endpoint = <&mipi_vc0_to_ipu1_csi1_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu1_csi1_mux_from_mipi_vc1: endpoint {
+				remote-endpoint = <&mipi_vc1_to_ipu1_csi1_mux>;
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu1_csi1_mux_from_mipi_vc2: endpoint {
+				remote-endpoint = <&mipi_vc2_to_ipu1_csi1_mux>;
+			};
+		};
+
+		port@3 {
+			reg = <3>;
+
+			ipu1_csi1_mux_from_mipi_vc3: endpoint {
+				remote-endpoint = <&mipi_vc3_to_ipu1_csi1_mux>;
+			};
+		};
+
+		port@4 {
+			reg = <4>;
+
+			ipu1_csi1_mux_from_parallel_sensor: endpoint {
+			};
+		};
+
+		port@5 {
+			reg = <5>;
+
+			ipu1_csi1_mux_to_ipu1_csi1: endpoint {
+				remote-endpoint = <&ipu1_csi1_from_ipu1_csi1_mux>;
+			};
+		};
+	};
 };
 
 &gpio1 {
@@ -184,3 +298,79 @@
 &vpu {
 	compatible = "fsl,imx6dl-vpu", "cnm,coda960";
 };
+
+&ipu1_csi1 {
+	port@0 {
+		reg = <0>;
+		ipu1_csi1_from_ipu1_csi1_mux: endpoint {
+			remote-endpoint = <&ipu1_csi1_mux_to_ipu1_csi1>;
+		};
+	};
+};
+
+&mipi_csi {
+	sink-ports = <1>;
+
+	port@0 {
+		reg = <0>;
+
+		mipi_csi_from_mipi_sensor: endpoint {
+		};
+	};
+
+	port@1 {
+		reg = <1>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_vc0_to_ipu1_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+		};
+
+		mipi_vc0_to_ipu1_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc0>;
+		};
+	};
+
+	port@2 {
+		reg = <2>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_vc1_to_ipu1_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc1>;
+		};
+
+		mipi_vc1_to_ipu1_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc1>;
+		};
+	};
+
+	port@3 {
+		reg = <3>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_vc2_to_ipu1_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc2>;
+		};
+
+		mipi_vc2_to_ipu1_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc2>;
+		};
+	};
+
+	port@4 {
+		reg = <4>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+
+		mipi_vc3_to_ipu1_csi0_mux: endpoint@0 {
+			remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc3>;
+		};
+
+		mipi_vc3_to_ipu1_csi1_mux: endpoint@1 {
+			remote-endpoint = <&ipu1_csi1_mux_from_mipi_vc3>;
+		};
+	};
+};
diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
index 2b261ba..a0602c5 100644
--- a/arch/arm/boot/dts/imx6q.dtsi
+++ b/arch/arm/boot/dts/imx6q.dtsi
@@ -142,11 +142,270 @@
 			resets = <&src 4>;
 
 			ipu2_csi0: ipu2_csi@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
 				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_csi0_from_mipi_vc2: endpoint {
+						remote-endpoint = <&mipi_vc2_to_ipu2_csi0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_csi0_to_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu2_smfc0_from_csi0>;
+					};
+					ipu2_csi0_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_from_csi0>;
+					};
+					ipu2_csi0_to_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu2_ic_prpenc_from_csi0>;
+					};
+				};
 			};
 
 			ipu2_csi1: ipu2_csi@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
 				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_csi1_from_ipu2_csi1_mux: endpoint {
+						remote-endpoint = <&ipu2_csi1_mux_to_ipu2_csi1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_csi1_to_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu2_smfc1_from_csi1>;
+					};
+					ipu2_csi1_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_from_csi1>;
+					};
+					ipu2_csi1_to_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu2_ic_prpenc_from_csi1>;
+					};
+				};
+			};
+
+			ipu2_smfc0: ipu2_smfc@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_smfc0_from_csi0: endpoint {
+						remote-endpoint = <&ipu2_csi0_to_smfc0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_smfc0_to_ic_pp1: pp-endpoint {
+						remote-endpoint = <&ipu2_ic_pp1_from_smfc0>;
+					};
+					ipu2_smfc0_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_from_smfc0>;
+					};
+					ipu2_smfc0_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_smfc0>;
+					};
+					ipu2_smfc0_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_smfc0>;
+					};
+				};
+			};
+
+			ipu2_smfc1: ipu2_smfc@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_smfc1_from_csi1: endpoint {
+						remote-endpoint = <&ipu2_csi1_to_smfc1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_smfc1_to_ic_pp2: pp-endpoint {
+						remote-endpoint = <&ipu2_ic_pp2_from_smfc1>;
+					};
+					ipu2_smfc1_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_from_smfc1>;
+					};
+					ipu2_smfc1_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_smfc1>;
+					};
+					ipu2_smfc1_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_smfc1>;
+					};
+				};
+			};
+
+			ipu2_ic_prpenc: ipu2_ic_prpenc@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_ic_prpenc_from_csi0: csi0-endpoint {
+						remote-endpoint = <&ipu2_csi0_to_ic_prpenc>;
+					};
+					ipu2_ic_prpenc_from_csi1: csi1-endpoint {
+						remote-endpoint = <&ipu2_csi1_to_ic_prpenc>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_ic_prpenc_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_ic_prpenc>;
+					};
+					ipu2_ic_prpenc_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_ic_prpenc>;
+					};
+				};
+			};
+
+			ipu2_ic_prpvf: ipu2_ic_prpvf@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_ic_prpvf_from_csi0: csi0-endpoint {
+						remote-endpoint = <&ipu2_csi0_to_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_from_csi1: csi1-endpoint {
+						remote-endpoint = <&ipu2_csi1_to_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu2_smfc0_to_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu2_smfc1_to_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_from_m2m1: m2m1-endpoint {
+						remote-endpoint = <&m2m1_to_ic_prpvf>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_ic_prpvf_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_to_ic_pp0: pp0-endpoint {
+						remote-endpoint = <&ipu2_ic_pp0_from_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_to_ic_pp1: pp1-endpoint {
+						remote-endpoint = <&ipu2_ic_pp1_from_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_to_ic_pp2: pp2-endpoint {
+						remote-endpoint = <&ipu2_ic_pp2_from_ic_prpvf>;
+					};
+					ipu2_ic_prpvf_to_m2m1: m2m1-endpoint {
+						remote-endpoint = <&m2m1_from_ic_prpvf>;
+					};
+				};
+			};
+
+			ipu2_ic_pp0: ipu2_ic_pp@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_ic_pp0_from_m2m1: m2m1-endpoint {
+						remote-endpoint = <&m2m1_to_ic_pp0>;
+					};
+					ipu2_ic_pp0_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_to_ic_pp0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_ic_pp0_to_m2m1: endpoint {
+						remote-endpoint = <&m2m1_from_ic_pp0>;
+					};
+				};
+			};
+
+			ipu2_ic_pp1: ipu2_ic_pp@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_ic_pp1_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu2_smfc0_to_ic_pp1>;
+					};
+					ipu2_ic_pp1_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_to_ic_pp1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_ic_pp1_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_ic_pp1>;
+					};
+					ipu2_ic_pp1_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_ic_pp1>;
+					};
+				};
+			};
+
+			ipu2_ic_pp2: ipu2_ic_pp@2 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <2>;
+
+				port@0 {
+					reg = <0>;
+					ipu2_ic_pp2_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu2_smfc1_to_ic_pp2>;
+					};
+					ipu2_ic_pp2_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu2_ic_prpvf_to_ic_pp2>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu2_ic_pp2_to_camif2: camif2-endpoint {
+						remote-endpoint = <&camif2_from_ic_pp2>;
+					};
+					ipu2_ic_pp2_to_camif3: camif3-endpoint {
+						remote-endpoint = <&camif3_from_ic_pp2>;
+					};
+				};
 			};
 
 			ipu2_di0: ipu2_di@0 {
@@ -207,6 +466,73 @@
 		compatible = "fsl,imx-gpu-subsystem";
 		cores = <&gpu_2d>, <&gpu_3d>, <&gpu_vg>;
 	};
+
+
+	ipu1_csi0_mux: ipu1_csi0_mux@4 {
+		compatible = "imx-video-mux";
+		reg = <0x04 0x80000>;
+		gpr = <&gpr>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <2>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu1_csi0_mux_from_mipi_vc0: endpoint {
+				remote-endpoint = <&mipi_vc0_to_ipu1_csi0_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu1_csi0_mux_from_parallel_sensor: endpoint {
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu1_csi0_mux_to_ipu1_csi0: endpoint {
+				remote-endpoint = <&ipu1_csi0_from_ipu1_csi0_mux>;
+			};
+		};
+	};
+
+	ipu2_csi1_mux: ipu2_csi1_mux@4 {
+		compatible = "imx-video-mux";
+		reg = <0x04 0x100000>;
+		gpr = <&gpr>;
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <2>;
+
+		port@0 {
+			reg = <0>;
+
+			ipu2_csi1_mux_from_mipi_vc3: endpoint {
+				remote-endpoint = <&mipi_vc3_to_ipu2_csi1_mux>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+
+			ipu2_csi1_mux_from_parallel_sensor: endpoint {
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+
+			ipu2_csi1_mux_to_ipu2_csi1: endpoint {
+				remote-endpoint = <&ipu2_csi1_from_ipu2_csi1_mux>;
+			};
+		};
+	};
 };
 
 &gpio1 {
@@ -266,6 +592,15 @@
 	};
 };
 
+&ipu1_csi1 {
+	port@0 {
+		reg = <0>;
+		ipu1_csi1_from_mipi_vc1: endpoint {
+			remote-endpoint = <&mipi_vc1_to_ipu1_csi1>;
+		};
+	};
+};
+
 &ldb {
 	clocks = <&clks IMX6QDL_CLK_LDB_DI0_SEL>, <&clks IMX6QDL_CLK_LDB_DI1_SEL>,
 		 <&clks IMX6QDL_CLK_IPU1_DI0_SEL>, <&clks IMX6QDL_CLK_IPU1_DI1_SEL>,
@@ -312,6 +647,49 @@
 	};
 };
 
+&mipi_csi {
+	sink-ports = <1>;
+
+	port@0 {
+		reg = <0>;
+
+		mipi_csi_from_mipi_sensor: endpoint {
+		};
+	};
+
+	port@1 {
+		reg = <1>;
+
+		mipi_vc0_to_ipu1_csi0_mux: endpoint {
+			remote-endpoint = <&ipu1_csi0_mux_from_mipi_vc0>;
+		};
+	};
+
+	port@2 {
+		reg = <2>;
+
+		mipi_vc1_to_ipu1_csi1: endpoint {
+			remote-endpoint = <&ipu1_csi1_from_mipi_vc1>;
+		};
+	};
+
+	port@3 {
+		reg = <3>;
+
+		mipi_vc2_to_ipu2_csi0: endpoint {
+			remote-endpoint = <&ipu2_csi0_from_mipi_vc2>;
+		};
+	};
+
+	port@4 {
+		reg = <4>;
+
+		mipi_vc3_to_ipu2_csi1_mux: endpoint {
+			remote-endpoint = <&ipu2_csi1_mux_from_mipi_vc3>;
+		};
+	};
+};
+
 &mipi_dsi {
 	ports {
 		port@2 {
@@ -335,3 +713,112 @@
 &vpu {
 	compatible = "fsl,imx6q-vpu", "cnm,coda960";
 };
+
+&media0 {
+	m2m1: m2m@1 {
+		compatible = "fsl,imx-media-mem2mem";
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <2>;
+		reg = <1>;
+
+		port@0 {
+			reg = <0>;
+		};
+
+		port@1 {
+			reg = <1>;
+			m2m1_from_ic_pp0: endpoint {
+				remote-endpoint = <&ipu2_ic_pp0_to_m2m1>;
+			};
+			m2m1_from_ic_prpvf: prpvf-endpoint {
+				remote-endpoint = <&ipu2_ic_prpvf_to_m2m1>;
+			};
+		};
+
+		port@2 {
+			reg = <2>;
+			m2m1_to_ic_prpvf: prpvf-endpoint {
+				remote-endpoint = <&ipu2_ic_prpvf_from_m2m1>;
+			};
+			m2m1_to_ic_pp0: pp0-endpoint {
+				remote-endpoint = <&ipu2_ic_pp0_from_m2m1>;
+			};
+		};
+
+		port@3 {
+			reg = <3>;
+		};
+	};
+
+	camif2: camif@2 {
+		compatible = "fsl,imx-media-camif";
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <1>;
+		reg = <2>;
+
+		port@0 {
+			reg = <0>;
+			camif2_from_smfc0: smfc0-endpoint {
+				remote-endpoint = <&ipu2_smfc0_to_camif2>;
+			};
+			camif2_from_smfc1: smfc1-endpoint {
+				remote-endpoint = <&ipu2_smfc1_to_camif2>;
+			};
+			camif2_from_ic_prpenc: prpenc-endpoint {
+				remote-endpoint = <&ipu2_ic_prpenc_to_camif2>;
+			};
+			camif2_from_ic_prpvf: prpvf-endpoint {
+				remote-endpoint = <&ipu2_ic_prpvf_to_camif2>;
+			};
+			camif2_from_ic_pp1: pp1-endpoint {
+				remote-endpoint = <&ipu2_ic_pp1_to_camif2>;
+			};
+			camif2_from_ic_pp2: pp2-endpoint {
+				remote-endpoint = <&ipu2_ic_pp2_to_camif2>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+		};
+	};
+
+	camif3: camif@3 {
+		compatible = "fsl,imx-media-camif";
+		#address-cells = <1>;
+		#size-cells = <0>;
+		status = "okay";
+		sink-ports = <1>;
+		reg = <3>;
+
+		port@0 {
+			reg = <0>;
+			camif3_from_smfc0: smfc0-endpoint {
+				remote-endpoint = <&ipu2_smfc0_to_camif3>;
+			};
+			camif3_from_smfc1: smfc1-endpoint {
+				remote-endpoint = <&ipu2_smfc1_to_camif3>;
+			};
+			camif3_from_ic_prpenc: prpenc-endpoint {
+				remote-endpoint = <&ipu2_ic_prpenc_to_camif3>;
+			};
+			camif3_from_ic_prpvf: prpvf-endpoint {
+				remote-endpoint = <&ipu2_ic_prpvf_to_camif3>;
+			};
+			camif3_from_ic_pp1: pp1-endpoint {
+				remote-endpoint = <&ipu2_ic_pp1_to_camif3>;
+			};
+			camif3_from_ic_pp2: pp2-endpoint {
+				remote-endpoint = <&ipu2_ic_pp2_to_camif3>;
+			};
+		};
+
+		port@1 {
+			reg = <1>;
+		};
+	};
+};
diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
index 2465187..ea1e2f3 100644
--- a/arch/arm/boot/dts/imx6qdl.dtsi
+++ b/arch/arm/boot/dts/imx6qdl.dtsi
@@ -1123,6 +1123,8 @@
 			mipi_csi: mipi@021dc000 {
 				compatible = "fsl,imx-mipi-csi2";
 				reg = <0x021dc000 0x4000>;
+				#address-cells = <1>;
+				#size-cells = <0>;
 				interrupts = <0 100 0x04>, <0 101 0x04>;
 				clocks = <&clks IMX6QDL_CLK_HSI_TX>,
 					 <&clks IMX6QDL_CLK_VIDEO_27M>,
@@ -1227,11 +1229,263 @@
 			resets = <&src 2>;
 
 			ipu1_csi0: ipu1_csi@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
 				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_csi0_from_ipu1_csi0_mux: endpoint {
+						remote-endpoint = <&ipu1_csi0_mux_to_ipu1_csi0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_csi0_to_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu1_smfc0_from_csi0>;
+					};
+					ipu1_csi0_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_from_csi0>;
+					};
+					ipu1_csi0_to_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu1_ic_prpenc_from_csi0>;
+					};
+				};
 			};
 
 			ipu1_csi1: ipu1_csi@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <1>;
+
+				port@1 {
+					reg = <1>;
+					ipu1_csi1_to_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu1_smfc1_from_csi1>;
+					};
+					ipu1_csi1_to_ic_prpvf: prendpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_from_csi1>;
+					};
+					ipu1_csi1_to_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu1_ic_prpenc_from_csi1>;
+					};
+				};
+			};
+
+			ipu1_smfc0: ipu1_smfc@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_smfc0_from_csi0: endpoint {
+						remote-endpoint = <&ipu1_csi0_to_smfc0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_smfc0_to_ic_pp1: pp-endpoint {
+						remote-endpoint = <&ipu1_ic_pp1_from_smfc0>;
+					};
+					ipu1_smfc0_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_from_smfc0>;
+					};
+					ipu1_smfc0_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_smfc0>;
+					};
+					ipu1_smfc0_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_smfc0>;
+					};
+				};
+			};
+
+			ipu1_smfc1: ipu1_smfc@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
 				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_smfc1_from_csi1: endpoint {
+						remote-endpoint = <&ipu1_csi1_to_smfc1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_smfc1_to_ic_pp2: pp-endpoint {
+						remote-endpoint = <&ipu1_ic_pp2_from_smfc1>;
+					};
+					ipu1_smfc1_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_from_smfc1>;
+					};
+					ipu1_smfc1_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_smfc1>;
+					};
+					ipu1_smfc1_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_smfc1>;
+					};
+				};
+			};
+
+			ipu1_ic_prpenc: ipu1_ic_prpenc@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_ic_prpenc_from_csi0: csi0-endpoint {
+						remote-endpoint = <&ipu1_csi0_to_ic_prpenc>;
+					};
+					ipu1_ic_prpenc_from_csi1: csi1-endpoint {
+						remote-endpoint = <&ipu1_csi1_to_ic_prpenc>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_ic_prpenc_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_ic_prpenc>;
+					};
+					ipu1_ic_prpenc_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_ic_prpenc>;
+					};
+				};
+			};
+
+			ipu1_ic_prpvf: ipu1_ic_prpvf@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_ic_prpvf_from_csi0: csi0-endpoint {
+						remote-endpoint = <&ipu1_csi0_to_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_from_csi1: csi1-endpoint {
+						remote-endpoint = <&ipu1_csi1_to_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu1_smfc0_to_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu1_smfc1_to_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_from_m2m0: m2m0-endpoint {
+						remote-endpoint = <&m2m0_to_ic_prpvf>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_ic_prpvf_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_to_ic_pp0: pp0-endpoint {
+						remote-endpoint = <&ipu1_ic_pp0_from_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_to_ic_pp1: pp1-endpoint {
+						remote-endpoint = <&ipu1_ic_pp1_from_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_to_ic_pp2: pp2-endpoint {
+						remote-endpoint = <&ipu1_ic_pp2_from_ic_prpvf>;
+					};
+					ipu1_ic_prpvf_to_m2m0: m2m0-endpoint {
+						remote-endpoint = <&m2m0_from_ic_prpvf>;
+					};
+				};
+			};
+
+			ipu1_ic_pp0: ipu1_ic_pp@0 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_ic_pp0_from_m2m0: m2m0-endpoint {
+						remote-endpoint = <&m2m0_to_ic_pp0>;
+					};
+					ipu1_ic_pp0_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_ic_pp0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_ic_pp0_to_m2m0: endpoint {
+						remote-endpoint = <&m2m0_from_ic_pp0>;
+					};
+				};
+			};
+
+			ipu1_ic_pp1: ipu1_ic_pp@1 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_ic_pp1_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu1_smfc0_to_ic_pp1>;
+					};
+					ipu1_ic_pp1_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_ic_pp1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_ic_pp1_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_ic_pp1>;
+					};
+					ipu1_ic_pp1_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_ic_pp1>;
+					};
+				};
+			};
+
+			ipu1_ic_pp2: ipu1_ic_pp@2 {
+				#address-cells = <1>;
+				#size-cells = <0>;
+				sink-ports = <1>;
+				reg = <2>;
+
+				port@0 {
+					reg = <0>;
+					ipu1_ic_pp2_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu1_smfc1_to_ic_pp2>;
+					};
+					ipu1_ic_pp2_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_ic_pp2>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+					ipu1_ic_pp2_to_camif0: camif0-endpoint {
+						remote-endpoint = <&camif0_from_ic_pp2>;
+					};
+					ipu1_ic_pp2_to_camif1: camif1-endpoint {
+						remote-endpoint = <&camif1_from_ic_pp2>;
+					};
+				};
 			};
 
 			ipu1_di0: ipu1_di@0 {
@@ -1284,5 +1538,119 @@
 				};
 			};
 		};
+
+		media0: media@0 {
+			compatible = "fsl,imx-media", "simple-bus";
+			#address-cells = <1>;
+			#size-cells = <0>;
+			status = "okay";
+
+			m2m0: m2m@0 {
+				compatible = "fsl,imx-media-mem2mem";
+				#address-cells = <1>;
+				#size-cells = <0>;
+				status = "okay";
+				sink-ports = <2>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+				};
+
+				port@1 {
+					reg = <1>;
+					m2m0_from_ic_pp0: endpoint {
+						remote-endpoint = <&ipu1_ic_pp0_to_m2m0>;
+					};
+					m2m0_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_m2m0>;
+					};
+				};
+
+				port@2 {
+					reg = <2>;
+					m2m0_to_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_from_m2m0>;
+					};
+					m2m0_to_ic_pp0: pp0-endpoint {
+						remote-endpoint = <&ipu1_ic_pp0_from_m2m0>;
+					};
+				};
+
+				port@3 {
+					reg = <3>;
+				};
+			};
+
+			camif0: camif@0 {
+				compatible = "fsl,imx-media-camif";
+				#address-cells = <1>;
+				#size-cells = <0>;
+				status = "okay";
+				sink-ports = <1>;
+				reg = <0>;
+
+				port@0 {
+					reg = <0>;
+					camif0_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu1_smfc0_to_camif0>;
+					};
+					camif0_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu1_smfc1_to_camif0>;
+					};
+					camif0_from_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu1_ic_prpenc_to_camif0>;
+					};
+					camif0_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_camif0>;
+					};
+					camif0_from_ic_pp1: pp1-endpoint {
+						remote-endpoint = <&ipu1_ic_pp1_to_camif0>;
+					};
+					camif0_from_ic_pp2: pp2-endpoint {
+						remote-endpoint = <&ipu1_ic_pp2_to_camif0>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+				};
+			};
+
+			camif1: camif@1 {
+				compatible = "fsl,imx-media-camif";
+				#address-cells = <1>;
+				#size-cells = <0>;
+				status = "okay";
+				sink-ports = <1>;
+				reg = <1>;
+
+				port@0 {
+					reg = <0>;
+					camif1_from_smfc0: smfc0-endpoint {
+						remote-endpoint = <&ipu1_smfc0_to_camif1>;
+					};
+					camif1_from_smfc1: smfc1-endpoint {
+						remote-endpoint = <&ipu1_smfc1_to_camif1>;
+					};
+					camif1_from_ic_prpenc: prpenc-endpoint {
+						remote-endpoint = <&ipu1_ic_prpenc_to_camif1>;
+					};
+					camif1_from_ic_prpvf: prpvf-endpoint {
+						remote-endpoint = <&ipu1_ic_prpvf_to_camif1>;
+					};
+					camif1_from_ic_pp1: pp1-endpoint {
+						remote-endpoint = <&ipu1_ic_pp1_to_camif1>;
+					};
+					camif1_from_ic_pp2: pp2-endpoint {
+						remote-endpoint = <&ipu1_ic_pp2_to_camif1>;
+					};
+				};
+
+				port@1 {
+					reg = <1>;
+				};
+			};
+		};
 	};
 };
-- 
2.7.4

[PATCH 04/12] ARM: dts: imx6-sabrelite: add OV5642 and OV5640 camera sensors

[Steve Longerbeam]

Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.
Both hang off the same i2c2 bus, so they require different (and non-
default) i2c slave addresses.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi. It is set to transmit over MIPI virtual channel 1.

Note there is a pin conflict with GPIO6. This pin functions as a power
input pin to the OV5642, but ENET uses it as the h/w workaround for
erratum ERR006687, to wake-up the ARM cores on normal RX and TX packet
done events (see 6261c4c8). So workaround 6261c4c8 is reverted here to
support the OV5642, and the "fsl,err006687-workaround-present" boolean
also must be removed. The result is that the CPUidle driver will no longer
allow entering the deep idle states on the sabrelite.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6dl-sabrelite.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabrelite.dts    |   6 ++
 arch/arm/boot/dts/imx6qdl-sabrelite.dtsi | 122 ++++++++++++++++++++++++++++++-
 3 files changed, 129 insertions(+), 4 deletions(-)

diff --git a/arch/arm/boot/dts/imx6dl-sabrelite.dts b/arch/arm/boot/dts/imx6dl-sabrelite.dts
index 0f06ca5..fec2524 100644
--- a/arch/arm/boot/dts/imx6dl-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6dl-sabrelite.dts
@@ -48,3 +48,8 @@
 	model = "Freescale i.MX6 DualLite SABRE Lite Board";
 	compatible = "fsl,imx6dl-sabrelite", "fsl,imx6dl";
 };
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+	data-lanes = <0 1>;
+	clock-lanes = <2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabrelite.dts b/arch/arm/boot/dts/imx6q-sabrelite.dts
index 66d10d8..9e2d26d 100644
--- a/arch/arm/boot/dts/imx6q-sabrelite.dts
+++ b/arch/arm/boot/dts/imx6q-sabrelite.dts
@@ -52,3 +52,9 @@
 &sata {
 	status = "okay";
 };
+
+&ipu1_csi1_from_mipi_vc1 {
+	data-lanes = <0 1>;
+	clock-lanes = <2>;
+};
+
diff --git a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
index 81dd6cd..d7fcb1a2 100644
--- a/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabrelite.dtsi
@@ -39,6 +39,8 @@
  *     FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
  *     OTHER DEALINGS IN THE SOFTWARE.
  */
+
+#include <dt-bindings/clock/imx6qdl-clock.h>
 #include <dt-bindings/gpio/gpio.h>
 #include <dt-bindings/input/input.h>
 
@@ -96,6 +98,15 @@
 		};
 	};
 
+	mipi_xclk: mipi_xclk {
+		compatible = "pwm-clock";
+		#clock-cells = <0>;
+		clock-frequency = <22000000>;
+		clock-output-names = "mipi_pwm3";
+		pwms = <&pwm3 0 45>; /* 1 / 45 ns = 22 MHz */
+		status = "okay";
+	};
+
 	gpio-keys {
 		compatible = "gpio-keys";
 		pinctrl-names = "default";
@@ -220,6 +231,22 @@
 	};
 };
 
+&ipu1_csi0_from_ipu1_csi0_mux {
+	bus-width = <8>;
+	data-shift = <12>; /* Lines 19:12 used */
+	hsync-active = <1>;
+	vync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+	remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+	pinctrl-names = "default";
+	pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
 &audmux {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_audmux>;
@@ -271,9 +298,6 @@
 	txd1-skew-ps = <0>;
 	txd2-skew-ps = <0>;
 	txd3-skew-ps = <0>;
-	interrupts-extended = <&gpio1 6 IRQ_TYPE_LEVEL_HIGH>,
-			      <&intc 0 119 IRQ_TYPE_LEVEL_HIGH>;
-	fsl,err006687-workaround-present;
 	status = "okay";
 };
 
@@ -302,6 +326,52 @@
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_i2c2>;
 	status = "okay";
+
+	camera: ov5642@42 {
+		compatible = "ovti,ov5642";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5642>;
+		clocks = <&clks IMX6QDL_CLK_CKO2>;
+		clock-names = "xclk";
+		reg = <0x42>;
+		xclk = <24000000>;
+		reset-gpios = <&gpio1 8 GPIO_ACTIVE_LOW>;
+		pwdn-gpios = <&gpio1 6 GPIO_ACTIVE_HIGH>;
+		gp-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>;
+
+		port {
+			ov5642_to_ipu1_csi0_mux: endpoint {
+				remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+				bus-width = <8>;
+				hsync-active = <1>;
+				vsync-active = <1>;
+			};
+		};
+	};
+
+	mipi_camera: ov5640@40 {
+		compatible = "ovti,ov5640_mipi";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		clocks = <&mipi_xclk>;
+		clock-names = "xclk";
+		reg = <0x40>;
+		xclk = <22000000>;
+		reset-gpios = <&gpio2 5 GPIO_ACTIVE_LOW>; /* NANDF_D5 */
+		pwdn-gpios = <&gpio6 9 GPIO_ACTIVE_HIGH>; /* NANDF_WP_B */
+
+		port {
+			#address-cells = <1>;
+			#size-cells = <0>;
+
+			ov5640_to_mipi_csi: endpoint@1 {
+				reg = <1>;
+				remote-endpoint = <&mipi_csi_from_mipi_sensor>;
+				data-lanes = <0 1>;
+				clock-lanes = <2>;
+			};
+		};
+	};
 };
 
 &i2c3 {
@@ -374,7 +444,6 @@
 				MX6QDL_PAD_RGMII_RX_CTL__RGMII_RX_CTL	0x1b030
 				/* Phy reset */
 				MX6QDL_PAD_EIM_D23__GPIO3_IO23		0x000b0
-				MX6QDL_PAD_GPIO_6__ENET_IRQ		0x000b1
 			>;
 		};
 
@@ -449,6 +518,39 @@
 			>;
 		};
 
+		pinctrl_ov5642: ov5642grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x80000000
+				MX6QDL_PAD_GPIO_6__GPIO1_IO06   0x80000000
+				MX6QDL_PAD_GPIO_8__GPIO1_IO08   0x80000000
+				MX6QDL_PAD_GPIO_3__CCM_CLKO2    0x80000000
+			>;
+		};
+
+		pinctrl_ipu1_csi0: ipu1grp-csi0 {
+			fsl,pins = <
+				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12    0x80000000
+				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13    0x80000000
+				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14    0x80000000
+				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15    0x80000000
+				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16    0x80000000
+				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17    0x80000000
+				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18    0x80000000
+				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19    0x80000000
+				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK   0x80000000
+				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC      0x80000000
+				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC     0x80000000
+				MX6QDL_PAD_CSI0_DATA_EN__IPU1_CSI0_DATA_EN 0x80000000
+			>;
+		};
+
+                pinctrl_ov5640: ov5640grp {
+                        fsl,pins = <
+				MX6QDL_PAD_NANDF_D5__GPIO2_IO05 0x000b0
+				MX6QDL_PAD_NANDF_WP_B__GPIO6_IO09 0x0b0b0
+                        >;
+                };
+
 		pinctrl_pwm1: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_SD1_DAT3__PWM1_OUT 0x1b0b1
@@ -605,3 +707,15 @@
 	vmmc-supply = <&reg_3p3v>;
 	status = "okay";
 };
+
+&mipi_csi {
+        status = "okay";
+};
+
+/* Incoming port from sensor */
+&mipi_csi_from_mipi_sensor {
+        remote-endpoint = <&ov5640_to_mipi_csi>;
+        data-lanes = <0 1>;
+        clock-lanes = <2>;
+};
+
-- 
2.7.4

[PATCH 05/12] ARM: dts: imx6-sabresd: add OV5642 and OV5640 camera sensors

[Steve Longerbeam]

Enables the OV5642 parallel-bus sensor, and the OV5640 MIPI CSI-2 sensor.

The OV5642 connects to the parallel-bus mux input port on ipu1_csi0_mux.

The OV5640 connects to the input port on the MIPI CSI-2 receiver on
mipi_csi. It is set to transmit over MIPI virtual channel 1.

Until the OV5652 sensor module compatible with the SabreSD becomes
available for testing, the ov5642 node is currently disabled.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6dl-sabresd.dts   |   5 ++
 arch/arm/boot/dts/imx6q-sabresd.dts    |   5 ++
 arch/arm/boot/dts/imx6qdl-sabresd.dtsi | 114 ++++++++++++++++++++++++++++++++-
 3 files changed, 123 insertions(+), 1 deletion(-)

diff --git a/arch/arm/boot/dts/imx6dl-sabresd.dts b/arch/arm/boot/dts/imx6dl-sabresd.dts
index 1e45f2f..6cf7a50 100644
--- a/arch/arm/boot/dts/imx6dl-sabresd.dts
+++ b/arch/arm/boot/dts/imx6dl-sabresd.dts
@@ -15,3 +15,8 @@
 	model = "Freescale i.MX6 DualLite SABRE Smart Device Board";
 	compatible = "fsl,imx6dl-sabresd", "fsl,imx6dl";
 };
+
+&ipu1_csi1_from_ipu1_csi1_mux {
+	data-lanes = <0 1>;
+	clock-lanes = <2>;
+};
diff --git a/arch/arm/boot/dts/imx6q-sabresd.dts b/arch/arm/boot/dts/imx6q-sabresd.dts
index 9cbdfe7..8c1d7ad 100644
--- a/arch/arm/boot/dts/imx6q-sabresd.dts
+++ b/arch/arm/boot/dts/imx6q-sabresd.dts
@@ -23,3 +23,8 @@
 &sata {
 	status = "okay";
 };
+
+&ipu1_csi1_from_mipi_vc1 {
+	data-lanes = <0 1>;
+	clock-lanes = <2>;
+};
diff --git a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
index 8e9e0d9..e36e1e7 100644
--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -10,6 +10,7 @@
  * http://www.gnu.org/copyleft/gpl.html
  */
 
+#include <dt-bindings/clock/imx6qdl-clock.h>
 #include <dt-bindings/gpio/gpio.h>
 #include <dt-bindings/input/input.h>
 
@@ -146,6 +147,33 @@
 	};
 };
 
+&ipu1_csi0_from_ipu1_csi0_mux {
+	bus-width = <8>;
+	data-shift = <12>; /* Lines 19:12 used */
+	hsync-active = <1>;
+	vsync-active = <1>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+	remote-endpoint = <&ov5642_to_ipu1_csi0_mux>;
+};
+
+&ipu1_csi0 {
+	pinctrl-names = "default";
+	pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
+&mipi_csi {
+	status = "okay";
+};
+
+/* Incoming port from sensor */
+&mipi_csi_from_mipi_sensor {
+	remote-endpoint = <&ov5640_to_mipi_csi>;
+	data-lanes = <0 1>;
+	clock-lanes = <2>;
+};
+
 &audmux {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_audmux>;
@@ -214,7 +242,33 @@
 			0x8014 /* 4:FN_DMICCDAT */
 			0x0000 /* 5:Default */
 		>;
-       };
+	};
+
+	camera: ov5642@3c {
+		compatible = "ovti,ov5642";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5642>;
+		clocks = <&clks IMX6QDL_CLK_CKO>;
+		clock-names = "xclk";
+		reg = <0x3c>;
+		xclk = <24000000>;
+		DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+		AVDD-supply = <&vgen5_reg>;  /* 2.8v, rev C board is VGEN3
+						rev B board is VGEN5 */
+		DVDD-supply = <&vgen2_reg>;  /* 1.5v*/
+		pwdn-gpios = <&gpio1 16 GPIO_ACTIVE_HIGH>; /* SD1_DAT0 */
+		reset-gpios = <&gpio1 17 GPIO_ACTIVE_LOW>; /* SD1_DAT1 */
+		status = "disabled";
+
+		port {
+			ov5642_to_ipu1_csi0_mux: endpoint {
+				remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+				bus-width = <8>;
+				hsync-active = <1>;
+				vsync-active = <1>;
+			};
+		};
+	};
 };
 
 &i2c2 {
@@ -322,6 +376,34 @@
 			};
 		};
 	};
+
+	mipi_camera: ov5640@3c {
+		compatible = "ovti,ov5640_mipi";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		reg = <0x3c>;
+		clocks = <&clks IMX6QDL_CLK_CKO>;
+		clock-names = "xclk";
+		xclk = <24000000>;
+		DOVDD-supply = <&vgen4_reg>; /* 1.8v */
+		AVDD-supply = <&vgen5_reg>;  /* 2.8v, rev C board is VGEN3
+						rev B board is VGEN5 */
+		DVDD-supply = <&vgen2_reg>;  /* 1.5v*/
+		pwdn-gpios = <&gpio1 19 GPIO_ACTIVE_HIGH>; /* SD1_DAT2 */
+		reset-gpios = <&gpio1 20 GPIO_ACTIVE_LOW>; /* SD1_CLK */
+
+		port {
+			#address-cells = <1>;
+			#size-cells = <0>;
+
+			ov5640_to_mipi_csi: endpoint@1 {
+				reg = <1>;
+				remote-endpoint = <&mipi_csi_from_mipi_sensor>;
+				data-lanes = <0 1>;
+				clock-lanes = <2>;
+			};
+		};
+	};
 };
 
 &i2c3 {
@@ -426,6 +508,36 @@
 			>;
 		};
 
+		pinctrl_ov5640: ov5640grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT2__GPIO1_IO19 0x80000000
+				MX6QDL_PAD_SD1_CLK__GPIO1_IO20  0x80000000
+			>;
+		};
+
+		pinctrl_ov5642: ov5642grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPIO1_IO16 0x80000000
+				MX6QDL_PAD_SD1_DAT1__GPIO1_IO17 0x80000000
+			>;
+		};
+
+		pinctrl_ipu1_csi0: ipu1grp-csi0 {
+			fsl,pins = <
+				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12    0x80000000
+				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13    0x80000000
+				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14    0x80000000
+				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15    0x80000000
+				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16    0x80000000
+				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17    0x80000000
+				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18    0x80000000
+				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19    0x80000000
+				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK   0x80000000
+				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC      0x80000000
+				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC     0x80000000
+			>;
+		};
+
 		pinctrl_pcie: pciegrp {
 			fsl,pins = <
 				MX6QDL_PAD_GPIO_17__GPIO7_IO12	0x1b0b0
-- 
2.7.4

[PATCH 06/12] ARM: dts: imx6-sabreauto: create i2cmux for i2c3

[Steve Longerbeam]

The sabreauto uses a steering pin to select between the SDA signal on
i2c3 bus, and a data-in pin for an SPI NOR chip. Use i2cmux to control
this steering pin. Idle state of the i2cmux selects SPI NOR. This is not
a classic way to use i2cmux, since one side of the mux selects something
other than an i2c bus, but it works and is probably the cleanest
solution. Note that if one thread is attempting to access SPI NOR while
another thread is accessing i2c3, the SPI NOR access will fail since the
i2cmux has selected the SDA pin rather than SPI NOR data-in. This couldn't
be avoided in any case, the board is not designed to allow concurrent
i2c3 and SPI NOR functions (and the default device-tree does not enable
SPI NOR anyway).

Devices hanging off i2c3 should now be defined under i2cmux, so
that the steering pin can be properly controlled to access those
devices. The port expanders (MAX7310) are thus moved into i2cmux.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 65 +++++++++++++++++++++-----------
 1 file changed, 44 insertions(+), 21 deletions(-)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index e000e6f..043d20c 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -108,6 +108,44 @@
 		default-brightness-level = <7>;
 		status = "okay";
 	};
+
+	i2cmux {
+		compatible = "i2c-mux-gpio";
+		#address-cells = <1>;
+		#size-cells = <0>;
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_i2c3mux>;
+		mux-gpios = <&gpio5 4 0>;
+		i2c-parent = <&i2c3>;
+		idle-state = <0>;
+
+		i2c@1 {
+			#address-cells = <1>;
+			#size-cells = <0>;
+			reg = <1>;
+
+			max7310_a: gpio@30 {
+				compatible = "maxim,max7310";
+				reg = <0x30>;
+				gpio-controller;
+				#gpio-cells = <2>;
+			};
+
+			max7310_b: gpio@32 {
+				compatible = "maxim,max7310";
+				reg = <0x32>;
+				gpio-controller;
+				#gpio-cells = <2>;
+			};
+
+			max7310_c: gpio@34 {
+				compatible = "maxim,max7310";
+				reg = <0x34>;
+				gpio-controller;
+				#gpio-cells = <2>;
+			};
+		};
+	};
 };
 
 &clks {
@@ -291,27 +329,6 @@
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_i2c3>;
 	status = "okay";
-
-	max7310_a: gpio@30 {
-		compatible = "maxim,max7310";
-		reg = <0x30>;
-		gpio-controller;
-		#gpio-cells = <2>;
-	};
-
-	max7310_b: gpio@32 {
-		compatible = "maxim,max7310";
-		reg = <0x32>;
-		gpio-controller;
-		#gpio-cells = <2>;
-	};
-
-	max7310_c: gpio@34 {
-		compatible = "maxim,max7310";
-		reg = <0x34>;
-		gpio-controller;
-		#gpio-cells = <2>;
-	};
 };
 
 &iomuxc {
@@ -419,6 +436,12 @@
 			>;
 		};
 
+		pinctrl_i2c3mux: i2c3muxgrp {
+			fsl,pins = <
+				MX6QDL_PAD_EIM_A24__GPIO5_IO04 0x80000000
+			>;
+		};
+
 		pinctrl_pwm3: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_SD4_DAT1__PWM3_OUT		0x1b0b1
-- 
2.7.4

[PATCH 07/12] ARM: dts: imx6-sabreauto: add reset-gpios property for max7310_b

[Steve Longerbeam]

The reset pin to the port expander chip (MAX7310) is controlled by a gpio,
so define a reset-gpios property to control it. There are three MAX7310's
on the SabreAuto CPU card (max7310_[abc]), but all use the same pin for
their reset. Since all can't acquire the same pin, assign it to max7310_b,
that chip is needed by more functions (usb and adv7180).

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 043d20c..944a4fa 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -136,6 +136,9 @@
 				reg = <0x32>;
 				gpio-controller;
 				#gpio-cells = <2>;
+				pinctrl-names = "default";
+				pinctrl-0 = <&pinctrl_max7310>;
+				reset-gpios = <&gpio1 15 GPIO_ACTIVE_LOW>;
 			};
 
 			max7310_c: gpio@34 {
@@ -442,6 +445,12 @@
 			>;
 		};
 
+		pinctrl_max7310: max7310grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD2_DAT0__GPIO1_IO15 0x80000000
+			>;
+		};
+
 		pinctrl_pwm3: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_SD4_DAT1__PWM3_OUT		0x1b0b1
-- 
2.7.4

[PATCH 08/12] ARM: dts: imx6-sabreauto: add pinctrl for gpt input capture

[Steve Longerbeam]

Add pinctrl groups for both GPT input capture channels.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index 944a4fa..d74882a 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -457,6 +457,18 @@
 			>;
 		};
 
+		pinctrl_gpt_input_capture0: gptinputcapture0grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT0__GPT_CAPTURE1	0x80000000
+			>;
+		};
+
+		pinctrl_gpt_input_capture1: gptinputcapture1grp {
+			fsl,pins = <
+				MX6QDL_PAD_SD1_DAT1__GPT_CAPTURE2	0x80000000
+			>;
+		};
+
 		pinctrl_spdif: spdifgrp {
 			fsl,pins = <
 				MX6QDL_PAD_KEY_COL3__SPDIF_IN 0x1b0b0
-- 
2.7.4

[PATCH 09/12] ARM: dts: imx6-sabreauto: add the ADV7180 video decoder

[Steve Longerbeam]

Enables the ADV7180 decoder sensor. The ADV7180 connects to the
parallel-bus mux input on ipu1_csi0_mux.

On the sabreauto, two analog video inputs are routed to the ADV7180,
composite on Ain1, and composite on Ain3. Those inputs are defined
via inputs and input-names under the ADV7180 node. The ADV7180 power
pin is via max7310_b port expander.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 arch/arm/boot/dts/imx6qdl-sabreauto.dtsi | 62 ++++++++++++++++++++++++++++++++
 1 file changed, 62 insertions(+)

diff --git a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
index d74882a..9222026 100644
--- a/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabreauto.dtsi
@@ -147,10 +147,48 @@
 				gpio-controller;
 				#gpio-cells = <2>;
 			};
+
+			camera: adv7180@21 {
+				compatible = "adi,adv7180";
+				reg = <0x21>;
+				powerdown-gpios = <&max7310_b 2 GPIO_ACTIVE_LOW>;
+				interrupt-parent = <&gpio1>;
+				interrupts = <27 0x8>;
+				inputs = <0x00 0x02>;
+				input-names = "ADV7180 Composite on Ain1",
+						"ADV7180 Composite on Ain3";
+
+				port {
+					adv7180_to_ipu1_csi0_mux: endpoint {
+						remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
+						bus-width = <8>;
+					};
+				};
+			};
 		};
 	};
 };
 
+&ipu1_smfc0 {
+	fim {
+		enable = <1>;
+	};
+};
+
+&ipu1_csi0_from_ipu1_csi0_mux {
+	bus-width = <8>;
+};
+
+&ipu1_csi0_mux_from_parallel_sensor {
+	remote-endpoint = <&adv7180_to_ipu1_csi0_mux>;
+	bus-width = <8>;
+};
+
+&ipu1_csi0 {
+	pinctrl-names = "default";
+	pinctrl-0 = <&pinctrl_ipu1_csi0>;
+};
+
 &clks {
 	assigned-clocks = <&clks IMX6QDL_PLL4_BYPASS_SRC>,
 			  <&clks IMX6QDL_PLL4_BYPASS>,
@@ -451,6 +489,30 @@
 			>;
 		};
 
+		pinctrl_ipu1_csi0: ipu1grp-csi0 {
+			fsl,pins = <
+				MX6QDL_PAD_CSI0_DAT4__IPU1_CSI0_DATA04   0x80000000
+				MX6QDL_PAD_CSI0_DAT5__IPU1_CSI0_DATA05   0x80000000
+				MX6QDL_PAD_CSI0_DAT6__IPU1_CSI0_DATA06   0x80000000
+				MX6QDL_PAD_CSI0_DAT7__IPU1_CSI0_DATA07   0x80000000
+				MX6QDL_PAD_CSI0_DAT8__IPU1_CSI0_DATA08   0x80000000
+				MX6QDL_PAD_CSI0_DAT9__IPU1_CSI0_DATA09   0x80000000
+				MX6QDL_PAD_CSI0_DAT10__IPU1_CSI0_DATA10  0x80000000
+				MX6QDL_PAD_CSI0_DAT11__IPU1_CSI0_DATA11  0x80000000
+				MX6QDL_PAD_CSI0_DAT12__IPU1_CSI0_DATA12  0x80000000
+				MX6QDL_PAD_CSI0_DAT13__IPU1_CSI0_DATA13  0x80000000
+				MX6QDL_PAD_CSI0_DAT14__IPU1_CSI0_DATA14  0x80000000
+				MX6QDL_PAD_CSI0_DAT15__IPU1_CSI0_DATA15  0x80000000
+				MX6QDL_PAD_CSI0_DAT16__IPU1_CSI0_DATA16  0x80000000
+				MX6QDL_PAD_CSI0_DAT17__IPU1_CSI0_DATA17  0x80000000
+				MX6QDL_PAD_CSI0_DAT18__IPU1_CSI0_DATA18  0x80000000
+				MX6QDL_PAD_CSI0_DAT19__IPU1_CSI0_DATA19  0x80000000
+				MX6QDL_PAD_CSI0_PIXCLK__IPU1_CSI0_PIXCLK 0x80000000
+				MX6QDL_PAD_CSI0_MCLK__IPU1_CSI0_HSYNC    0x80000000
+				MX6QDL_PAD_CSI0_VSYNC__IPU1_CSI0_VSYNC   0x80000000
+			>;
+		};
+
 		pinctrl_pwm3: pwm1grp {
 			fsl,pins = <
 				MX6QDL_PAD_SD4_DAT1__PWM3_OUT		0x1b0b1
-- 
2.7.4

[PATCH 10/12] gpu: ipu-v3: Add ipu_unit_type enumeration

[Steve Longerbeam]

Adds an enumeration of the major IPUv3 subunits. Provide that info
in struct ipu_client_platformdata to more easily determine the IPU
client type.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 include/video/imx-ipu-v3.h | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/include/video/imx-ipu-v3.h b/include/video/imx-ipu-v3.h
index 53cd07c..7709af7 100644
--- a/include/video/imx-ipu-v3.h
+++ b/include/video/imx-ipu-v3.h
@@ -28,6 +28,23 @@ enum ipuv3_type {
 	IPUV3H,
 };
 
+/*
+ * Enumeration of the major IPU subunits
+ */
+enum ipu_unit_type {
+	IPU_IDMAC = 0,
+	IPU_CM,
+	IPU_CSI,
+	IPU_SMFC,
+	IPU_IC,
+	IPU_VDI,
+	IPU_IRT,
+	IPU_DC,
+	IPU_DI,
+	IPU_DP,
+	IPU_DMFC,
+};
+
 #define IPU_PIX_FMT_GBR24	v4l2_fourcc('G', 'B', 'R', '3')
 
 /*
@@ -399,6 +416,7 @@ int ipu_rot_mode_to_degrees(int *degrees, enum ipu_rotate_mode mode,
 			    bool hflip, bool vflip);
 
 struct ipu_client_platformdata {
+	enum ipu_unit_type type;
 	int csi;
 	int di;
 	int dc;
-- 
2.7.4

[PATCH 11/12] gpu: ipu-v3: lookup ipu client nodes by name

[Steve Longerbeam]

To allow for IPU clients containing multiple ports, they are no longer
a single port node name, but have a name of the format
"ipu<id>_<unit>". So we can no longer use of_graph_get_port_by_id()
to lookup the client node.

Create the function of_get_ipu_client_node() that looks up the client
node by node name and unit id. The ipu_unit_type enumeration is added
to the client_reg[] entries to compose the node names.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/gpu/ipu-v3/ipu-common.c | 55 +++++++++++++++++++++++++++++++++++------
 1 file changed, 47 insertions(+), 8 deletions(-)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index 97218af..b6ca36b 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -1159,6 +1159,7 @@ struct ipu_platform_reg {
 static struct ipu_platform_reg client_reg[] = {
 	{
 		.pdata = {
+			.type = IPU_CSI,
 			.csi = 0,
 			.dma[0] = IPUV3_CHANNEL_CSI0,
 			.dma[1] = -EINVAL,
@@ -1166,6 +1167,7 @@ static struct ipu_platform_reg client_reg[] = {
 		.name = "imx-ipuv3-csi",
 	}, {
 		.pdata = {
+			.type = IPU_CSI,
 			.csi = 1,
 			.dma[0] = IPUV3_CHANNEL_CSI1,
 			.dma[1] = -EINVAL,
@@ -1173,6 +1175,7 @@ static struct ipu_platform_reg client_reg[] = {
 		.name = "imx-ipuv3-csi",
 	}, {
 		.pdata = {
+			.type = IPU_DI,
 			.di = 0,
 			.dc = 5,
 			.dp = IPU_DP_FLOW_SYNC_BG,
@@ -1182,6 +1185,7 @@ static struct ipu_platform_reg client_reg[] = {
 		.name = "imx-ipuv3-crtc",
 	}, {
 		.pdata = {
+			.type = IPU_DI,
 			.di = 1,
 			.dc = 1,
 			.dp = -EINVAL,
@@ -1195,6 +1199,46 @@ static struct ipu_platform_reg client_reg[] = {
 static DEFINE_MUTEX(ipu_client_id_mutex);
 static int ipu_client_id;
 
+static struct device_node *
+of_get_ipu_client_node(struct ipu_soc *ipu, struct ipu_platform_reg *reg)
+{
+	struct device *dev = ipu->dev;
+	struct device_node *client;
+	char node_name[32];
+	u32 id, client_id = 0;
+
+	switch (reg->pdata.type) {
+	case IPU_CSI:
+		snprintf(node_name, sizeof(node_name),
+			 "ipu%d_csi", ipu->id + 1);
+		client_id = reg->pdata.csi;
+		break;
+	case IPU_DI:
+		snprintf(node_name, sizeof(node_name),
+			 "ipu%d_di", ipu->id + 1);
+		client_id = reg->pdata.di;
+		break;
+	default:
+		client = NULL;
+		goto out;
+	}
+
+	for_each_child_of_node(dev->of_node, client) {
+		if (client->name &&
+		    (of_node_cmp(client->name, node_name) == 0)) {
+			of_property_read_u32(client, "reg", &id);
+			if (id == client_id)
+				break;
+		}
+	}
+out:
+	if (!client)
+		dev_info(dev, "no %s%d node in %s, not using %s%d\n",
+			 node_name, client_id, dev->of_node->full_name,
+			 node_name, client_id);
+	return client;
+}
+
 static int ipu_add_client_devices(struct ipu_soc *ipu, unsigned long ipu_base)
 {
 	struct device *dev = ipu->dev;
@@ -1211,15 +1255,10 @@ static int ipu_add_client_devices(struct ipu_soc *ipu, unsigned long ipu_base)
 		struct platform_device *pdev;
 		struct device_node *of_node;
 
-		/* Associate subdevice with the corresponding port node */
-		of_node = of_graph_get_port_by_id(dev->of_node, i);
-		if (!of_node) {
-			dev_info(dev,
-				 "no port@%d node in %s, not using %s%d\n",
-				 i, dev->of_node->full_name,
-				 (i / 2) ? "DI" : "CSI", i % 2);
+		/* Associate subdevice with the corresponding client node */
+		of_node = of_get_ipu_client_node(ipu, reg);
+		if (!of_node)
 			continue;
-		}
 
 		pdev = platform_device_alloc(reg->name, id++);
 		if (!pdev) {
-- 
2.7.4

[PATCH 12/12] gpu: ipu-v3: Add smfc and ic client devices

[Steve Longerbeam]

Adds IPU client devices for the SMFC and IC task units.

Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
---
 drivers/gpu/ipu-v3/ipu-common.c | 87 +++++++++++++++++++++++++++++++++++++++--
 include/video/imx-ipu-v3.h      |  3 ++
 2 files changed, 87 insertions(+), 3 deletions(-)

diff --git a/drivers/gpu/ipu-v3/ipu-common.c b/drivers/gpu/ipu-v3/ipu-common.c
index b6ca36b..729581d 100644
--- a/drivers/gpu/ipu-v3/ipu-common.c
+++ b/drivers/gpu/ipu-v3/ipu-common.c
@@ -1161,18 +1161,77 @@ static struct ipu_platform_reg client_reg[] = {
 		.pdata = {
 			.type = IPU_CSI,
 			.csi = 0,
-			.dma[0] = IPUV3_CHANNEL_CSI0,
-			.dma[1] = -EINVAL,
+			.dma[0] = -EINVAL,
 		},
 		.name = "imx-ipuv3-csi",
 	}, {
 		.pdata = {
 			.type = IPU_CSI,
 			.csi = 1,
+			.dma[0] = -EINVAL,
+		},
+		.name = "imx-ipuv3-csi",
+	}, {
+		.pdata = {
+			.type = IPU_SMFC,
+			.smfc = 0,
+			.dma[0] = IPUV3_CHANNEL_CSI0,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-smfc",
+	}, {
+		.pdata = {
+			.type = IPU_SMFC,
+			.smfc = 1,
 			.dma[0] = IPUV3_CHANNEL_CSI1,
 			.dma[1] = -EINVAL,
 		},
-		.name = "imx-ipuv3-csi",
+		.name = "imx-ipuv3-smfc",
+	}, {
+		.pdata = {
+			.type = IPU_IC,
+			.ic_task = IC_TASK_ENCODER,
+			.ic = 0,
+			.dma[0] = IPUV3_CHANNEL_IC_PRP_ENC_MEM,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-ic",
+	}, {
+		.pdata = {
+			.type = IPU_IC,
+			.ic_task = IC_TASK_VIEWFINDER,
+			.ic = 0,
+			.dma[0] = IPUV3_CHANNEL_IC_PRP_VF_MEM,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-ic",
+	}, {
+		.pdata = {
+			.type = IPU_IC,
+			.ic_task = IC_TASK_POST_PROCESSOR,
+			.ic = 0,
+			.dma[0] = IPUV3_CHANNEL_IC_PP_MEM,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-ic",
+	}, {
+		.pdata = {
+			.type = IPU_IC,
+			.ic_task = IC_TASK_POST_PROCESSOR,
+			.ic = 1,
+			.dma[0] = IPUV3_CHANNEL_IC_PP_MEM,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-ic",
+	}, {
+		.pdata = {
+			.type = IPU_IC,
+			.ic_task = IC_TASK_POST_PROCESSOR,
+			.ic = 2,
+			.dma[0] = IPUV3_CHANNEL_IC_PP_MEM,
+			.dma[1] = -EINVAL,
+		},
+		.name = "imx-ipuv3-ic",
 	}, {
 		.pdata = {
 			.type = IPU_DI,
@@ -1213,6 +1272,28 @@ of_get_ipu_client_node(struct ipu_soc *ipu, struct ipu_platform_reg *reg)
 			 "ipu%d_csi", ipu->id + 1);
 		client_id = reg->pdata.csi;
 		break;
+	case IPU_SMFC:
+		snprintf(node_name, sizeof(node_name), "ipu%d_smfc",
+			 ipu->id + 1);
+		client_id = reg->pdata.smfc;
+		break;
+	case IPU_IC:
+		switch (reg->pdata.ic_task) {
+		case IC_TASK_ENCODER:
+			snprintf(node_name, sizeof(node_name),
+				 "ipu%d_ic_prpenc", ipu->id + 1);
+			break;
+		case IC_TASK_VIEWFINDER:
+			snprintf(node_name, sizeof(node_name),
+				 "ipu%d_ic_prpvf", ipu->id + 1);
+			break;
+		case IC_TASK_POST_PROCESSOR:
+			snprintf(node_name, sizeof(node_name),
+				 "ipu%d_ic_pp", ipu->id + 1);
+			break;
+		}
+		client_id = reg->pdata.ic;
+		break;
 	case IPU_DI:
 		snprintf(node_name, sizeof(node_name),
 			 "ipu%d_di", ipu->id + 1);
diff --git a/include/video/imx-ipu-v3.h b/include/video/imx-ipu-v3.h
index 7709af7..4e70ca4 100644
--- a/include/video/imx-ipu-v3.h
+++ b/include/video/imx-ipu-v3.h
@@ -418,6 +418,9 @@ int ipu_rot_mode_to_degrees(int *degrees, enum ipu_rotate_mode mode,
 struct ipu_client_platformdata {
 	enum ipu_unit_type type;
 	int csi;
+	int smfc;
+	int ic_task;
+	int ic;
 	int di;
 	int dc;
 	int dp;
-- 
2.7.4

Re: [PATCH 02/12] ARM: dts: imx6qdl: rename ipu client nodes

[Philipp Zabel]

Hi Steve,

Am Mittwoch, den 07.12.2016, 16:57 -0800 schrieb Steve Longerbeam:
> To allow for IPU client devices that are composed of more than one
> port for input and output (SMFC and IC), change the nodes from being
> a single port node to nodes that can contain multiple ports. Rename
> the nodes to use the following format: "ipu<id>_<subunit>".

Don't do this. These are not IPU client device nodes, they are input and
output port nodes according to the of graph bindings, as described in
Documentation/devicetree/bindings/display/imx/fsl-imx-drm.txt.

They happen to match to the CSI0/1 and DI0/1 modules, but really what is
described here are the external connections of the IPU. The IPU
internals are not described in the device tree at all.

regards
Philipp

> The IPUv3 driver will then need to lookup the client nodes by name
> rather than by port id.
> 
> Signed-off-by: Steve Longerbeam <steve_longerbeam@mentor.com>
> ---
>  arch/arm/boot/dts/imx6q.dtsi   | 12 ++++++------
>  arch/arm/boot/dts/imx6qdl.dtsi | 12 ++++++------
>  2 files changed, 12 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/arm/boot/dts/imx6q.dtsi b/arch/arm/boot/dts/imx6q.dtsi
> index e9a5d0b..2b261ba 100644
> --- a/arch/arm/boot/dts/imx6q.dtsi
> +++ b/arch/arm/boot/dts/imx6q.dtsi
> @@ -141,18 +141,18 @@
>  			clock-names = "bus", "di0", "di1";
>  			resets = <&src 4>;
>  
> -			ipu2_csi0: port@0 {
> +			ipu2_csi0: ipu2_csi@0 {
>  				reg = <0>;
>  			};
>  
> -			ipu2_csi1: port@1 {
> +			ipu2_csi1: ipu2_csi@1 {
>  				reg = <1>;
>  			};
>  
> -			ipu2_di0: port@2 {
> +			ipu2_di0: ipu2_di@0 {
>  				#address-cells = <1>;
>  				#size-cells = <0>;
> -				reg = <2>;
> +				reg = <0>;
>  
>  				ipu2_di0_disp0: disp0-endpoint {
>  				};
> @@ -174,10 +174,10 @@
>  				};
>  			};
>  
> -			ipu2_di1: port@3 {
> +			ipu2_di1: ipu2_di@1 {
>  				#address-cells = <1>;
>  				#size-cells = <0>;
> -				reg = <3>;
> +				reg = <1>;
>  
>  				ipu2_di1_hdmi: hdmi-endpoint {
>  					remote-endpoint = <&hdmi_mux_3>;
> diff --git a/arch/arm/boot/dts/imx6qdl.dtsi b/arch/arm/boot/dts/imx6qdl.dtsi
> index e01e5d5..2465187 100644
> --- a/arch/arm/boot/dts/imx6qdl.dtsi
> +++ b/arch/arm/boot/dts/imx6qdl.dtsi
> @@ -1226,18 +1226,18 @@
>  			clock-names = "bus", "di0", "di1";
>  			resets = <&src 2>;
>  
> -			ipu1_csi0: port@0 {
> +			ipu1_csi0: ipu1_csi@0 {
>  				reg = <0>;
>  			};
>  
> -			ipu1_csi1: port@1 {
> +			ipu1_csi1: ipu1_csi@1 {
>  				reg = <1>;
>  			};
>  
> -			ipu1_di0: port@2 {
> +			ipu1_di0: ipu1_di@0 {
>  				#address-cells = <1>;
>  				#size-cells = <0>;
> -				reg = <2>;
> +				reg = <0>;
>  
>  				ipu1_di0_disp0: disp0-endpoint {
>  				};
> @@ -1259,10 +1259,10 @@
>  				};
>  			};
>  
> -			ipu1_di1: port@3 {
> +			ipu1_di1: ipu1_di@1 {
>  				#address-cells = <1>;
>  				#size-cells = <0>;
> -				reg = <3>;
> +				reg = <1>;
>  
>  				ipu1_di1_disp1: disp1-endpoint {
>  				};