[RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
use an OV5640 camera.

The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.

With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
pipeline can be configured with the following:

    media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"

The camera and various nodes in the pipeline can be configured for UYVY:
    media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
    media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"

With that, the media pipeline looks like:


Media controller API version 5.15.0

Media device information
------------------------
driver          imx7-csi
model           imx-media
serial          
bus info        platform:32e20000.csi
hw revision     0x0
driver version  5.15.0

Device topology
- entity 1: csi (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		<- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
	pad1: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
		-> "csi capture":0 [ENABLED,IMMUTABLE]

- entity 4: csi capture (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "csi":1 [ENABLED,IMMUTABLE]

- entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev1
	pad0: Sink
		[fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
		<- "ov5640 1-003c":0 [ENABLED]
	pad1: Source
		[fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
		-> "csi":0 [ENABLED,IMMUTABLE]

- entity 15: ov5640 1-003c (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
		-> "imx7-mipi-csis.0":0 [ENABLED]

When configured, gstreamer can be used to capture 1 frame and store it to a file.

gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test

Unfortunately, the video capture never appears to happen.  No errors occur, not
interrupts are recorded and no errors are recorded.

gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
PLAYING ...
New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
Clock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709


If anyone has any insight as to what might be wrong, I'd like feedback.
I posted a device tree that I beleive goes with the newer imx8mm-evk, but
I do not have this hardware, so I cannot test it.

Adam Ford (5):
  arm64: dts: imx8mm: Add CSI nodes
  arm64: defconfig: Enable VIDEO_IMX_MEDIA
  arm64: dts: imx8mm-beacon: Enable OV5640 Camera
  arm64: defconfig: Enable OV5640
  arm64: dts: imx8mm-evk: Enable OV5640 Camera

 .../freescale/imx8mm-beacon-baseboard.dtsi    | 55 +++++++++++++++++++
 arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi | 44 +++++++++++++++
 arch/arm64/boot/dts/freescale/imx8mm.dtsi     | 55 +++++++++++++++++++
 arch/arm64/configs/defconfig                  |  2 +
 4 files changed, 156 insertions(+)

-- 
2.25.1

[RFC V2 1/5] arm64: dts: imx8mm: Add CSI nodes

[Adam Ford]

There is a csi bridge and csis interface that tie together
to allow csi2 capture.

Signed-off-by: Adam Ford <aford173@gmail.com>
---
 arch/arm64/boot/dts/freescale/imx8mm.dtsi | 55 +++++++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/imx8mm.dtsi b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
index c2f3f118f82e..920f9041ef50 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
@@ -1068,6 +1068,22 @@ aips4: bus@32c00000 {
 			#size-cells = <1>;
 			ranges = <0x32c00000 0x32c00000 0x400000>;
 
+			csi: csi@32e20000 {
+				compatible = "fsl,imx8mm-csi", "fsl,imx7-csi";
+				reg = <0x32e20000 0x1000>;
+				interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
+				clocks = <&clk IMX8MM_CLK_CSI1_ROOT>;
+				clock-names = "mclk";
+				power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_CSI_BRIDGE>;
+				status = "disabled";
+
+				port {
+					csi_in: endpoint {
+						remote-endpoint = <&imx8mm_mipi_csi_out>;
+					};
+				};
+			};
+
 			disp_blk_ctrl: blk-ctrl@32e28000 {
 				compatible = "fsl,imx8mm-disp-blk-ctrl", "syscon";
 				reg = <0x32e28000 0x100>;
@@ -1095,6 +1111,45 @@ disp_blk_ctrl: blk-ctrl@32e28000 {
 				#power-domain-cells = <1>;
 			};
 
+			mipi_csi2: mipi-csi@32e30000 {
+				compatible = "fsl,imx8mm-mipi-csi2";
+				reg = <0x32e30000 0x1000>;
+				interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
+				assigned-clocks = <&clk IMX8MM_CLK_CSI1_CORE>,
+						  <&clk IMX8MM_CLK_CSI1_PHY_REF>;
+				assigned-clock-parents = <&clk IMX8MM_SYS_PLL2_1000M>,
+							  <&clk IMX8MM_SYS_PLL2_1000M>;
+				clock-frequency = <333000000>;
+				clocks = <&clk IMX8MM_CLK_DISP_APB_ROOT>,
+					 <&clk IMX8MM_CLK_CSI1_ROOT>,
+					 <&clk IMX8MM_CLK_CSI1_PHY_REF>,
+					 <&clk IMX8MM_CLK_DISP_AXI_ROOT>;
+				clock-names = "pclk", "wrap", "phy", "axi";
+				power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_MIPI_CSI>;
+				status = "disabled";
+
+				ports {
+					#address-cells = <1>;
+					#size-cells = <0>;
+
+					port@0 {
+						reg = <0>;
+
+						imx8mm_mipi_csi_in: endpoint {
+						};
+					};
+
+					port@1 {
+						reg = <1>;
+
+						imx8mm_mipi_csi_out: endpoint {
+							remote-endpoint = <&csi_in>;
+						};
+					};
+				};
+			};
+
+
 			usbotg1: usb@32e40000 {
 				compatible = "fsl,imx8mm-usb", "fsl,imx7d-usb";
 				reg = <0x32e40000 0x200>;
-- 
2.25.1

[RFC V2 2/5] arm64: defconfig: Enable VIDEO_IMX_MEDIA

[Adam Ford]

To use a camera, the CSIS and CSI drivers need to be enabled with
VIDEO_IMX_MEDIA.

Signed-off-by: Adam Ford <aford173@gmail.com>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index d04cdfc7daa4..6be8fb8248d5 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -955,6 +955,7 @@ CONFIG_MFD_CROS_EC_DEV=y
 CONFIG_STAGING=y
 CONFIG_STAGING_MEDIA=y
 CONFIG_VIDEO_HANTRO=m
+CONFIG_VIDEO_IMX_MEDIA=m
 CONFIG_CHROME_PLATFORMS=y
 CONFIG_CROS_EC=y
 CONFIG_CROS_EC_I2C=y
-- 
2.25.1

[RFC V2 3/5] arm64: dts: imx8mm-beacon: Enable OV5640 Camera

[Adam Ford]

The baseboard has support for a TDNext 5640 Camera which
uses an OV5640 connected to a 2-lane CSI2 interface.

Signed-off-by: Adam Ford <aford173@gmail.com>
---
 .../freescale/imx8mm-beacon-baseboard.dtsi    | 55 +++++++++++++++++++
 1 file changed, 55 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
index 6f5e63696ec0..cb19c80ac29c 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-beacon-baseboard.dtsi
@@ -43,6 +43,16 @@ reg_audio: regulator-audio {
 		enable-active-high;
 	};
 
+	reg_camera: regulator-camera {
+		compatible = "regulator-fixed";
+		regulator-name = "mipi_pwr";
+		regulator-min-microvolt = <2800000>;
+		regulator-max-microvolt = <2800000>;
+		gpio = <&pca6416_1 0 GPIO_ACTIVE_HIGH>;
+		enable-active-high;
+		startup-delay-us = <100000>;
+	};
+
 	reg_usdhc2_vmmc: regulator-usdhc2 {
 		compatible = "regulator-fixed";
 		regulator-name = "VSD_3V3";
@@ -67,6 +77,10 @@ sound {
 	};
 };
 
+&csi {
+	status = "okay";
+};
+
 &ecspi2 {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_espi2>;
@@ -90,6 +104,30 @@ &i2c2 {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_i2c2>;
 	status = "okay";
+
+	camera@3c {
+		compatible = "ovti,ov5640";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		reg = <0x3c>;
+		clocks = <&clk IMX8MM_CLK_CLKO1>;
+		clock-names = "xclk";
+		assigned-clocks = <&clk IMX8MM_CLK_CLKO1>;
+		assigned-clock-parents = <&clk IMX8MM_CLK_24M>;
+		assigned-clock-rates = <24000000>;
+		AVDD-supply = <&reg_camera>;  /* 2.8v */
+		powerdown-gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
+		reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
+
+		port {
+			/* MIPI CSI-2 bus endpoint */
+			ov5640_to_mipi_csi2: endpoint {
+				remote-endpoint = <&imx8mm_mipi_csi_in>;
+				clock-lanes = <0>;
+				data-lanes = <1 2>;
+			};
+		};
+	};
 };
 
 &i2c4 {
@@ -141,6 +179,15 @@ pca6416_1: gpio@21 {
 	};
 };
 
+&imx8mm_mipi_csi_in {
+	remote-endpoint = <&ov5640_to_mipi_csi2>;
+	data-lanes = <1 2>;
+};
+
+&mipi_csi2 {
+	status = "okay";
+};
+
 &sai3 {
 	pinctrl-names = "default";
 	pinctrl-0 = <&pinctrl_sai3>;
@@ -209,6 +256,14 @@ MX8MM_IOMUXC_SAI3_RXFS_GPIO4_IO28	0x41
 		>;
 	};
 
+	pinctrl_ov5640: ov5640grp {
+		fsl,pins = <
+			MX8MM_IOMUXC_GPIO1_IO07_GPIO1_IO7		0x19
+			MX8MM_IOMUXC_GPIO1_IO06_GPIO1_IO6		0x19
+			MX8MM_IOMUXC_GPIO1_IO14_CCMSRCGPCMIX_CLKO1	0x59
+		>;
+	};
+
 	pinctrl_pcal6414: pcal6414-gpiogrp {
 		fsl,pins = <
 			MX8MM_IOMUXC_SAI2_MCLK_GPIO4_IO27		0x19
-- 
2.25.1

[RFC V2 4/5] arm64: defconfig: Enable OV5640

[Adam Ford]

The Beacon EmbeddedWorks imx8mm development kit has a TD Next 5640
Camera.  Enable the OV5640 driver to use the camera.

Signed-off-by: Adam Ford <aford173@gmail.com>
---
 arch/arm64/configs/defconfig | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/arm64/configs/defconfig b/arch/arm64/configs/defconfig
index 6be8fb8248d5..6030fa71bcbd 100644
--- a/arch/arm64/configs/defconfig
+++ b/arch/arm64/configs/defconfig
@@ -665,6 +665,7 @@ CONFIG_VIDEO_QCOM_VENUS=m
 CONFIG_SDR_PLATFORM_DRIVERS=y
 CONFIG_VIDEO_RCAR_DRIF=m
 CONFIG_VIDEO_IMX219=m
+CONFIG_VIDEO_OV5640=m
 CONFIG_VIDEO_OV5645=m
 CONFIG_VIDEO_QCOM_CAMSS=m
 CONFIG_DRM=m
-- 
2.25.1

[RFC V2 5/5] arm64: dts: imx8mm-evk: Enable OV5640 Camera

[Adam Ford]

The schematic shows support for a camera interface, and the NXP
kernel shows it is an OV5640.

Signed-off-by: Adam Ford <aford173@gmail.com>
---
 arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi | 44 +++++++++++++++++++
 1 file changed, 44 insertions(+)

diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
index e033d0257b5a..27217d30b8d8 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
@@ -239,6 +239,10 @@ ldo6_reg: LDO6 {
 	};
 };
 
+&csi {
+	status = "okay";
+};
+
 &i2c2 {
 	clock-frequency = <400000>;
 	pinctrl-names = "default";
@@ -287,6 +291,38 @@ pca6416: gpio@20 {
 		gpio-controller;
 		#gpio-cells = <2>;
 	};
+
+	camera@3c {
+		compatible = "ovti,ov5640";
+		pinctrl-names = "default";
+		pinctrl-0 = <&pinctrl_ov5640>;
+		reg = <0x3c>;
+		clocks = <&clk IMX8MM_CLK_CLKO1>;
+		clock-names = "xclk";
+		assigned-clocks = <&clk IMX8MM_CLK_CLKO1>;
+		assigned-clock-parents = <&clk IMX8MM_CLK_24M>;
+		assigned-clock-rates = <24000000>;
+		powerdown-gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
+		reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
+
+		port {
+			/* MIPI CSI-2 bus endpoint */
+			ov5640_to_mipi_csi2: endpoint {
+				remote-endpoint = <&imx8mm_mipi_csi_in>;
+				clock-lanes = <0>;
+				data-lanes = <1 2>;
+			};
+		};
+	};
+};
+
+&imx8mm_mipi_csi_in {
+	remote-endpoint = <&ov5640_to_mipi_csi2>;
+	data-lanes = <1 2>;
+};
+
+&mipi_csi2 {
+	status = "okay";
 };
 
 &sai3 {
@@ -406,6 +442,14 @@ MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA			0x400001c3
 		>;
 	};
 
+	pinctrl_ov5640: ov5640grp {
+		fsl,pins = <
+			MX8MM_IOMUXC_GPIO1_IO07_GPIO1_IO7		0x19
+			MX8MM_IOMUXC_GPIO1_IO06_GPIO1_IO6		0x19
+			MX8MM_IOMUXC_GPIO1_IO14_CCMSRCGPCMIX_CLKO1	0x59
+		>;
+	};
+
 	pinctrl_pmic: pmicirqgrp {
 		fsl,pins = <
 			MX8MM_IOMUXC_GPIO1_IO03_GPIO1_IO3		0x141
-- 
2.25.1

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Fabio Estevam]

Hi Adam,

[Adding Frieder on Cc]

On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
>
> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> use an OV5640 camera.
>
> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
>
> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> pipeline can be configured with the following:
>
>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
>
> The camera and various nodes in the pipeline can be configured for UYVY:
>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
>
> With that, the media pipeline looks like:
>
>
> Media controller API version 5.15.0
>
> Media device information
> ------------------------
> driver          imx7-csi
> model           imx-media
> serial
> bus info        platform:32e20000.csi
> hw revision     0x0
> driver version  5.15.0
>
> Device topology
> - entity 1: csi (2 pads, 2 links)
>             type V4L2 subdev subtype Unknown flags 0
>             device node name /dev/v4l-subdev0
>         pad0: Sink
>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
>         pad1: Source
>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
>
> - entity 4: csi capture (1 pad, 1 link)
>             type Node subtype V4L flags 0
>             device node name /dev/video0
>         pad0: Sink
>                 <- "csi":1 [ENABLED,IMMUTABLE]
>
> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
>              type V4L2 subdev subtype Unknown flags 0
>              device node name /dev/v4l-subdev1
>         pad0: Sink
>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>                 <- "ov5640 1-003c":0 [ENABLED]
>         pad1: Source
>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>                 -> "csi":0 [ENABLED,IMMUTABLE]
>
> - entity 15: ov5640 1-003c (1 pad, 1 link)
>              type V4L2 subdev subtype Sensor flags 0
>              device node name /dev/v4l-subdev2
>         pad0: Source
>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
>                 -> "imx7-mipi-csis.0":0 [ENABLED]
>
> When configured, gstreamer can be used to capture 1 frame and store it to a file.
>
> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
>
> Unfortunately, the video capture never appears to happen.  No errors occur, not
> interrupts are recorded and no errors are recorded.
>
> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Pipeline is PREROLLED ...
> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> PLAYING ...
> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> Clock
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
>
>
> If anyone has any insight as to what might be wrong, I'd like feedback.
> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> I do not have this hardware, so I cannot test it.

It seems that Frieder on Cc managed to get camera capture to work on
i.MX8MM here:
https://git.kontron-electronics.de/sw/misc/linux/-/commits/v5.10-mx8mm-csi

Hopefully, this can help to figure out what is missing in mainline to
get camera capture to work on i.MX8M.

I don't have access to an OV5640 camera to connect to the imx8mm-evk
board to try your series.

Regards,

Fabio Estevam

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi Adam,
>
> [Adding Frieder on Cc]
>
> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> >
> > The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > use an OV5640 camera.
> >
> > The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> >
> > With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > pipeline can be configured with the following:
> >
> >     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> >
> > The camera and various nodes in the pipeline can be configured for UYVY:
> >     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> >     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> >
> > With that, the media pipeline looks like:
> >
> >
> > Media controller API version 5.15.0
> >
> > Media device information
> > ------------------------
> > driver          imx7-csi
> > model           imx-media
> > serial
> > bus info        platform:32e20000.csi
> > hw revision     0x0
> > driver version  5.15.0
> >
> > Device topology
> > - entity 1: csi (2 pads, 2 links)
> >             type V4L2 subdev subtype Unknown flags 0
> >             device node name /dev/v4l-subdev0
> >         pad0: Sink
> >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> >                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> >         pad1: Source
> >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> >                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> >
> > - entity 4: csi capture (1 pad, 1 link)
> >             type Node subtype V4L flags 0
> >             device node name /dev/video0
> >         pad0: Sink
> >                 <- "csi":1 [ENABLED,IMMUTABLE]
> >
> > - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> >              type V4L2 subdev subtype Unknown flags 0
> >              device node name /dev/v4l-subdev1
> >         pad0: Sink
> >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> >                 <- "ov5640 1-003c":0 [ENABLED]
> >         pad1: Source
> >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> >                 -> "csi":0 [ENABLED,IMMUTABLE]
> >
> > - entity 15: ov5640 1-003c (1 pad, 1 link)
> >              type V4L2 subdev subtype Sensor flags 0
> >              device node name /dev/v4l-subdev2
> >         pad0: Source
> >                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> >                 -> "imx7-mipi-csis.0":0 [ENABLED]
> >
> > When configured, gstreamer can be used to capture 1 frame and store it to a file.
> >
> > gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> >
> > Unfortunately, the video capture never appears to happen.  No errors occur, not
> > interrupts are recorded and no errors are recorded.
> >
> > gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > Setting pipeline to PAUSED ...
> > Pipeline is live and does not need PREROLL ...
> > Pipeline is PREROLLED ...
> > Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > PLAYING ...
> > New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > Clock
> > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> >
> >
> > If anyone has any insight as to what might be wrong, I'd like feedback.
> > I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > I do not have this hardware, so I cannot test it.
>
> It seems that Frieder on Cc managed to get camera capture to work on
> i.MX8MM here:
> https://git.kontron-electronics.de/sw/misc/linux/-/commits/v5.10-mx8mm-csi
>
> Hopefully, this can help to figure out what is missing in mainline to
> get camera capture to work on i.MX8M.
>
> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> board to try your series.

Fabio,

Thanks for the heads up on that repo.  I was able to use that repo and
get still images to capture on an OV5640, but I noticed a fair amount
of differences between that repo and what's found in linux-next.

Laurent,

I haven't exhausted the patch differences, but I found at least a few
that appear to be missiing upstream, and I am curious to know if/what
your opinion is on whether or not they're needed, since the patches on
Frieder's repo appear to come from you.
[1] - media: imx: imx7-media-csi: Add i.MX8MM identification
[2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
[3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats

media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats

Maybe these don't need to be applied, but they are 'some' of the
differences that I see between this 5.10 branch and linux-next .  I
know there are more, but


[1] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/8ac7ec6db0c260a871038721886dbdb6660ed84c
[2] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/0b5727c8eba8c370f7db5eace0243f78992a4dbb
[3] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/14befa6bc146b10092a6ac5d0ed4d42c87c6cf27

Frieder et al,

Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
am curious to see if a regression popped in somewhere, but git bisect
will make this difficult since there is a fair amount of variation
between this custom repo and the upstream.

adam
>
> Regards,
>
> Fabio Estevam

Re: [RFC V2 5/5] arm64: dts: imx8mm-evk: Enable OV5640 Camera

[Laurent Pinchart]

Hi Adam,

Thank you for the patch.

On Sat, Oct 23, 2021 at 03:34:56PM -0500, Adam Ford wrote:
> The schematic shows support for a camera interface, and the NXP
> kernel shows it is an OV5640.

The camera is an external module though. Should this be a DT overlay ?

> Signed-off-by: Adam Ford <aford173@gmail.com>
> ---
>  arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi | 44 +++++++++++++++++++
>  1 file changed, 44 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> index e033d0257b5a..27217d30b8d8 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> @@ -239,6 +239,10 @@ ldo6_reg: LDO6 {
>  	};
>  };
>  
> +&csi {
> +	status = "okay";
> +};
> +
>  &i2c2 {
>  	clock-frequency = <400000>;
>  	pinctrl-names = "default";
> @@ -287,6 +291,38 @@ pca6416: gpio@20 {
>  		gpio-controller;
>  		#gpio-cells = <2>;
>  	};
> +
> +	camera@3c {
> +		compatible = "ovti,ov5640";
> +		pinctrl-names = "default";
> +		pinctrl-0 = <&pinctrl_ov5640>;
> +		reg = <0x3c>;
> +		clocks = <&clk IMX8MM_CLK_CLKO1>;
> +		clock-names = "xclk";
> +		assigned-clocks = <&clk IMX8MM_CLK_CLKO1>;
> +		assigned-clock-parents = <&clk IMX8MM_CLK_24M>;
> +		assigned-clock-rates = <24000000>;
> +		powerdown-gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
> +		reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
> +
> +		port {
> +			/* MIPI CSI-2 bus endpoint */
> +			ov5640_to_mipi_csi2: endpoint {
> +				remote-endpoint = <&imx8mm_mipi_csi_in>;
> +				clock-lanes = <0>;
> +				data-lanes = <1 2>;
> +			};
> +		};
> +	};
> +};
> +
> +&imx8mm_mipi_csi_in {
> +	remote-endpoint = <&ov5640_to_mipi_csi2>;
> +	data-lanes = <1 2>;
> +};
> +
> +&mipi_csi2 {
> +	status = "okay";
>  };
>  
>  &sai3 {
> @@ -406,6 +442,14 @@ MX8MM_IOMUXC_I2C3_SDA_I2C3_SDA			0x400001c3
>  		>;
>  	};
>  
> +	pinctrl_ov5640: ov5640grp {
> +		fsl,pins = <
> +			MX8MM_IOMUXC_GPIO1_IO07_GPIO1_IO7		0x19
> +			MX8MM_IOMUXC_GPIO1_IO06_GPIO1_IO6		0x19
> +			MX8MM_IOMUXC_GPIO1_IO14_CCMSRCGPCMIX_CLKO1	0x59
> +		>;
> +	};
> +
>  	pinctrl_pmic: pmicirqgrp {
>  		fsl,pins = <
>  			MX8MM_IOMUXC_GPIO1_IO03_GPIO1_IO3		0x141

-- 
Regards,

Laurent Pinchart

Re: [RFC V2 1/5] arm64: dts: imx8mm: Add CSI nodes

[Laurent Pinchart]

Hi Adam,

Thank you for the patch.

On Sat, Oct 23, 2021 at 03:34:52PM -0500, Adam Ford wrote:
> There is a csi bridge and csis interface that tie together
> to allow csi2 capture.
> 
> Signed-off-by: Adam Ford <aford173@gmail.com>
> ---
>  arch/arm64/boot/dts/freescale/imx8mm.dtsi | 55 +++++++++++++++++++++++
>  1 file changed, 55 insertions(+)
> 
> diff --git a/arch/arm64/boot/dts/freescale/imx8mm.dtsi b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
> index c2f3f118f82e..920f9041ef50 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mm.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8mm.dtsi
> @@ -1068,6 +1068,22 @@ aips4: bus@32c00000 {
>  			#size-cells = <1>;
>  			ranges = <0x32c00000 0x32c00000 0x400000>;
>  
> +			csi: csi@32e20000 {
> +				compatible = "fsl,imx8mm-csi", "fsl,imx7-csi";
> +				reg = <0x32e20000 0x1000>;
> +				interrupts = <GIC_SPI 16 IRQ_TYPE_LEVEL_HIGH>;
> +				clocks = <&clk IMX8MM_CLK_CSI1_ROOT>;
> +				clock-names = "mclk";
> +				power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_CSI_BRIDGE>;
> +				status = "disabled";
> +
> +				port {
> +					csi_in: endpoint {
> +						remote-endpoint = <&imx8mm_mipi_csi_out>;
> +					};
> +				};
> +			};
> +
>  			disp_blk_ctrl: blk-ctrl@32e28000 {
>  				compatible = "fsl,imx8mm-disp-blk-ctrl", "syscon";
>  				reg = <0x32e28000 0x100>;
> @@ -1095,6 +1111,45 @@ disp_blk_ctrl: blk-ctrl@32e28000 {
>  				#power-domain-cells = <1>;
>  			};
>  
> +			mipi_csi2: mipi-csi@32e30000 {

I'd rename the label to mipi_csi to match the name in the reference
manual.

> +				compatible = "fsl,imx8mm-mipi-csi2";
> +				reg = <0x32e30000 0x1000>;
> +				interrupts = <GIC_SPI 17 IRQ_TYPE_LEVEL_HIGH>;
> +				assigned-clocks = <&clk IMX8MM_CLK_CSI1_CORE>,
> +						  <&clk IMX8MM_CLK_CSI1_PHY_REF>;
> +				assigned-clock-parents = <&clk IMX8MM_SYS_PLL2_1000M>,
> +							  <&clk IMX8MM_SYS_PLL2_1000M>;
> +				clock-frequency = <333000000>;
> +				clocks = <&clk IMX8MM_CLK_DISP_APB_ROOT>,
> +					 <&clk IMX8MM_CLK_CSI1_ROOT>,
> +					 <&clk IMX8MM_CLK_CSI1_PHY_REF>,
> +					 <&clk IMX8MM_CLK_DISP_AXI_ROOT>;
> +				clock-names = "pclk", "wrap", "phy", "axi";
> +				power-domains = <&disp_blk_ctrl IMX8MM_DISPBLK_PD_MIPI_CSI>;
> +				status = "disabled";
> +
> +				ports {
> +					#address-cells = <1>;
> +					#size-cells = <0>;
> +
> +					port@0 {
> +						reg = <0>;
> +
> +						imx8mm_mipi_csi_in: endpoint {
> +						};

Empty ports are fine as they model the hardware, even when not
connected, but an endpoint models a connection, so it needs a remote
endpoint. You can drop the endpoint here, board DT files will create one
with

&mipi_csi {
	ports {
		port@0 {
			board_endpoint: endpoint {
				...
			};
		};
	};
};

> +					};
> +
> +					port@1 {
> +						reg = <1>;
> +
> +						imx8mm_mipi_csi_out: endpoint {
> +							remote-endpoint = <&csi_in>;
> +						};
> +					};
> +				};
> +			};
> +
> +

A single blank line is enough.

Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

>  			usbotg1: usb@32e40000 {
>  				compatible = "fsl,imx8mm-usb", "fsl,imx7d-usb";
>  				reg = <0x32e40000 0x200>;

-- 
Regards,

Laurent Pinchart

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Laurent Pinchart]

Hi Adam,

On Wed, Oct 27, 2021 at 07:39:40PM -0500, Adam Ford wrote:
> On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam wrote:
> >
> > Hi Adam,
> >
> > [Adding Frieder on Cc]
> >
> > On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > >
> > > The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > > those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > > use an OV5640 camera.
> > >
> > > The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > > of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > > they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > >
> > > With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > > pipeline can be configured with the following:
> > >
> > >     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > >
> > > The camera and various nodes in the pipeline can be configured for UYVY:
> > >     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > >     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > >
> > > With that, the media pipeline looks like:
> > >
> > >
> > > Media controller API version 5.15.0
> > >
> > > Media device information
> > > ------------------------
> > > driver          imx7-csi
> > > model           imx-media
> > > serial
> > > bus info        platform:32e20000.csi
> > > hw revision     0x0
> > > driver version  5.15.0
> > >
> > > Device topology
> > > - entity 1: csi (2 pads, 2 links)
> > >             type V4L2 subdev subtype Unknown flags 0
> > >             device node name /dev/v4l-subdev0
> > >         pad0: Sink
> > >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > >                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > >         pad1: Source
> > >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > >                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > >
> > > - entity 4: csi capture (1 pad, 1 link)
> > >             type Node subtype V4L flags 0
> > >             device node name /dev/video0
> > >         pad0: Sink
> > >                 <- "csi":1 [ENABLED,IMMUTABLE]
> > >
> > > - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > >              type V4L2 subdev subtype Unknown flags 0
> > >              device node name /dev/v4l-subdev1
> > >         pad0: Sink
> > >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > >                 <- "ov5640 1-003c":0 [ENABLED]
> > >         pad1: Source
> > >                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > >                 -> "csi":0 [ENABLED,IMMUTABLE]
> > >
> > > - entity 15: ov5640 1-003c (1 pad, 1 link)
> > >              type V4L2 subdev subtype Sensor flags 0
> > >              device node name /dev/v4l-subdev2
> > >         pad0: Source
> > >                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > >                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > >
> > > When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > >
> > > gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > >
> > > Unfortunately, the video capture never appears to happen.  No errors occur, not
> > > interrupts are recorded and no errors are recorded.
> > >
> > > gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > Setting pipeline to PAUSED ...
> > > Pipeline is live and does not need PREROLL ...
> > > Pipeline is PREROLLED ...
> > > Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > > PLAYING ...
> > > New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > > Clock
> > > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > >
> > >
> > > If anyone has any insight as to what might be wrong, I'd like feedback.
> > > I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > > I do not have this hardware, so I cannot test it.
> >
> > It seems that Frieder on Cc managed to get camera capture to work on
> > i.MX8MM here:
> > https://git.kontron-electronics.de/sw/misc/linux/-/commits/v5.10-mx8mm-csi
> >
> > Hopefully, this can help to figure out what is missing in mainline to
> > get camera capture to work on i.MX8M.
> >
> > I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > board to try your series.
> 
> Fabio,
> 
> Thanks for the heads up on that repo.  I was able to use that repo and
> get still images to capture on an OV5640, but I noticed a fair amount
> of differences between that repo and what's found in linux-next.
> 
> Laurent,
> 
> I haven't exhausted the patch differences, but I found at least a few
> that appear to be missiing upstream, and I am curious to know if/what
> your opinion is on whether or not they're needed, since the patches on
> Frieder's repo appear to come from you.
> [1] - media: imx: imx7-media-csi: Add i.MX8MM identification

This shouldn't make any difference.

> [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM

This was based on my analysis of the reference manual, but it turned out
to be incorrect, the i.MX8MM CSI-2 receiver didn't work at all for me
with that patch.

> [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats

This one isn't clear to me yet. I'll have to retest, I think I had to
set dual pixel mode to capture RAW8 and RAW10 with the OV5640, but
setting it broke raw capture with a different sensor. More investigation
is needed. In any case, with or without that patch, you should get
interrupts from the CSIS as the dual pixel mode is related to the
interface with the CSI bridge.

> media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> 
> Maybe these don't need to be applied, but they are 'some' of the
> differences that I see between this 5.10 branch and linux-next .  I
> know there are more, but
> 
> 
> [1] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/8ac7ec6db0c260a871038721886dbdb6660ed84c
> [2] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/0b5727c8eba8c370f7db5eace0243f78992a4dbb
> [3] - https://git.kontron-electronics.de/sw/misc/linux/-/commit/14befa6bc146b10092a6ac5d0ed4d42c87c6cf27
> 
> Frieder et al,
> 
> Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> am curious to see if a regression popped in somewhere, but git bisect
> will make this difficult since there is a fair amount of variation
> between this custom repo and the upstream.

I'll try to run tests on mainline again, but it will have to wait a
couple of weeks I'm afraid.

-- 
Regards,

Laurent Pinchart

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Frieder Schrempf]

On 28.10.21 02:39, Adam Ford wrote:
> On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
>>
>> Hi Adam,
>>
>> [Adding Frieder on Cc]
>>
>> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
>>>
>>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
>>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
>>> use an OV5640 camera.
>>>
>>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
>>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
>>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
>>>
>>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
>>> pipeline can be configured with the following:
>>>
>>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
>>>
>>> The camera and various nodes in the pipeline can be configured for UYVY:
>>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
>>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
>>>
>>> With that, the media pipeline looks like:
>>>
>>>
>>> Media controller API version 5.15.0
>>>
>>> Media device information
>>> ------------------------
>>> driver          imx7-csi
>>> model           imx-media
>>> serial
>>> bus info        platform:32e20000.csi
>>> hw revision     0x0
>>> driver version  5.15.0
>>>
>>> Device topology
>>> - entity 1: csi (2 pads, 2 links)
>>>             type V4L2 subdev subtype Unknown flags 0
>>>             device node name /dev/v4l-subdev0
>>>         pad0: Sink
>>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
>>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
>>>         pad1: Source
>>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
>>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 4: csi capture (1 pad, 1 link)
>>>             type Node subtype V4L flags 0
>>>             device node name /dev/video0
>>>         pad0: Sink
>>>                 <- "csi":1 [ENABLED,IMMUTABLE]
>>>
>>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
>>>              type V4L2 subdev subtype Unknown flags 0
>>>              device node name /dev/v4l-subdev1
>>>         pad0: Sink
>>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>>                 <- "ov5640 1-003c":0 [ENABLED]
>>>         pad1: Source
>>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
>>>                 -> "csi":0 [ENABLED,IMMUTABLE]
>>>
>>> - entity 15: ov5640 1-003c (1 pad, 1 link)
>>>              type V4L2 subdev subtype Sensor flags 0
>>>              device node name /dev/v4l-subdev2
>>>         pad0: Source
>>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
>>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
>>>
>>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
>>>
>>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
>>>
>>> Unfortunately, the video capture never appears to happen.  No errors occur, not
>>> interrupts are recorded and no errors are recorded.
>>>
>>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
>>> Setting pipeline to PAUSED ...
>>> Pipeline is live and does not need PREROLL ...
>>> Pipeline is PREROLLED ...
>>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
>>> PLAYING ...
>>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
>>> Clock
>>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
>>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
>>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
>>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
>>>
>>>
>>> If anyone has any insight as to what might be wrong, I'd like feedback.
>>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
>>> I do not have this hardware, so I cannot test it.
>>
>> It seems that Frieder on Cc managed to get camera capture to work on
>> i.MX8MM here:
>> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
>>
>> Hopefully, this can help to figure out what is missing in mainline to
>> get camera capture to work on i.MX8M.
>>
>> I don't have access to an OV5640 camera to connect to the imx8mm-evk
>> board to try your series.
> 
> Fabio,
> 
> Thanks for the heads up on that repo.  I was able to use that repo and
> get still images to capture on an OV5640, but I noticed a fair amount
> of differences between that repo and what's found in linux-next.
> 
> Laurent,
> 
> I haven't exhausted the patch differences, but I found at least a few
> that appear to be missiing upstream, and I am curious to know if/what
> your opinion is on whether or not they're needed, since the patches on
> Frieder's repo appear to come from you.
> [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> 
> media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> 
> Maybe these don't need to be applied, but they are 'some' of the
> differences that I see between this 5.10 branch and linux-next .  I
> know there are more, but
> 
> 
> [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> 
> Frieder et al,
> 
> Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> am curious to see if a regression popped in somewhere, but git bisect
> will make this difficult since there is a fair amount of variation
> between this custom repo and the upstream.

No, I haven't done anything with CSI on a more recent kernel. And I only
used CSI with the tree above and the ADV7280M bridge. I don't have any
hardware with a sensor/camera.

In case you haven't seen this already, here is a thread with some notes
about my testing results:
https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Tim Harvey]

On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
<frieder.schrempf@kontron.de> wrote:
>
> On 28.10.21 02:39, Adam Ford wrote:
> > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> >>
> >> Hi Adam,
> >>
> >> [Adding Frieder on Cc]
> >>
> >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> >>>
> >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> >>> use an OV5640 camera.
> >>>
> >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> >>>
> >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> >>> pipeline can be configured with the following:
> >>>
> >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> >>>
> >>> The camera and various nodes in the pipeline can be configured for UYVY:
> >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> >>>
> >>> With that, the media pipeline looks like:
> >>>
> >>>
> >>> Media controller API version 5.15.0
> >>>
> >>> Media device information
> >>> ------------------------
> >>> driver          imx7-csi
> >>> model           imx-media
> >>> serial
> >>> bus info        platform:32e20000.csi
> >>> hw revision     0x0
> >>> driver version  5.15.0
> >>>
> >>> Device topology
> >>> - entity 1: csi (2 pads, 2 links)
> >>>             type V4L2 subdev subtype Unknown flags 0
> >>>             device node name /dev/v4l-subdev0
> >>>         pad0: Sink
> >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> >>>         pad1: Source
> >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 4: csi capture (1 pad, 1 link)
> >>>             type Node subtype V4L flags 0
> >>>             device node name /dev/video0
> >>>         pad0: Sink
> >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> >>>              type V4L2 subdev subtype Unknown flags 0
> >>>              device node name /dev/v4l-subdev1
> >>>         pad0: Sink
> >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> >>>                 <- "ov5640 1-003c":0 [ENABLED]
> >>>         pad1: Source
> >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> >>>
> >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> >>>              type V4L2 subdev subtype Sensor flags 0
> >>>              device node name /dev/v4l-subdev2
> >>>         pad0: Source
> >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> >>>
> >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> >>>
> >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> >>>
> >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> >>> interrupts are recorded and no errors are recorded.
> >>>
> >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> >>> Setting pipeline to PAUSED ...
> >>> Pipeline is live and does not need PREROLL ...
> >>> Pipeline is PREROLLED ...
> >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> >>> PLAYING ...
> >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> >>> Clock
> >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> >>>
> >>>
> >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> >>> I do not have this hardware, so I cannot test it.
> >>
> >> It seems that Frieder on Cc managed to get camera capture to work on
> >> i.MX8MM here:
> >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> >>
> >> Hopefully, this can help to figure out what is missing in mainline to
> >> get camera capture to work on i.MX8M.
> >>
> >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> >> board to try your series.
> >
> > Fabio,
> >
> > Thanks for the heads up on that repo.  I was able to use that repo and
> > get still images to capture on an OV5640, but I noticed a fair amount
> > of differences between that repo and what's found in linux-next.
> >
> > Laurent,
> >
> > I haven't exhausted the patch differences, but I found at least a few
> > that appear to be missiing upstream, and I am curious to know if/what
> > your opinion is on whether or not they're needed, since the patches on
> > Frieder's repo appear to come from you.
> > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> >
> > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> >
> > Maybe these don't need to be applied, but they are 'some' of the
> > differences that I see between this 5.10 branch and linux-next .  I
> > know there are more, but
> >
> >
> > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> >
> > Frieder et al,
> >
> > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > am curious to see if a regression popped in somewhere, but git bisect
> > will make this difficult since there is a fair amount of variation
> > between this custom repo and the upstream.
>
> No, I haven't done anything with CSI on a more recent kernel. And I only
> used CSI with the tree above and the ADV7280M bridge. I don't have any
> hardware with a sensor/camera.
>
> In case you haven't seen this already, here is a thread with some notes
> about my testing results:
> https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
>

For what it's worth I've got another test setup for IMX8MM CSI
capture. I have a Raspberry Pi Camera module v2 connected to an
imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
4-lane CSI connection.

Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
the 'blk-ctl series on top of 5.15 and adding support to my dt via:

commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
Author: Tim Harvey <tharvey@gateworks.com>
Date:   Wed Oct 27 15:45:23 2021 -0700

    arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2

    Add support for rpi camera module v2 which is an IMX219 8MP module:
     - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
     - has its own on-board 24MHz osc so no clock required from baseboard
     - pin 11 enables 1.8V and 2.8V LDO which is connected to
       GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
       support powerdown-gpios and using gpio1 as reset-gpios

    Signed-off-by: Tim Harvey <tharvey@gateworks.com>

diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
index 7b00b6b5bb38..b708c80d884b 100644
--- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
+++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
@@ -35,6 +35,13 @@
                };
        };

+       cam24m: cam24m {
+               compatible = "fixed-clock";
+               #clock-cells = <0>;
+               clock-frequency = <24000000>;
+               clock-output-names = "cam24m";
+       };
+
        pcie0_refclk: pcie0-refclk {
                compatible = "fixed-clock";
                #clock-cells = <0>;
@@ -100,6 +107,19 @@
        };
 };

+&csi {
+       status = "okay";
+};
+
+&imx8mm_mipi_csi_in {
+       remote-endpoint = <&imx219_to_mipi_csi2>;
+       data-lanes = <1 2>;
+};
+
+&mipi_csi2 {
+       status = "okay";
+};
+
 /* off-board header */
 &ecspi2 {
        pinctrl-names = "default";
@@ -132,6 +152,25 @@
        pinctrl-names = "default";
        pinctrl-0 = <&pinctrl_i2c3>;
        status = "okay";
+
+       imx219: sensor@10 {
+               compatible = "sony,imx219";
+               pinctrl-names = "default";
+               pinctrl-0 = <&pinctrl_imx219>;
+               reg = <0x10>;
+               clocks = <&cam24m>;
+               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
+
+               port {
+                       /* MIPI CSI-2 bus endpoint */
+                       imx219_to_mipi_csi2: endpoint {
+                               remote-endpoint = <&imx8mm_mipi_csi_in>;
+                               clock-lanes = <0>;
+                               data-lanes = <1 2>;
+                               link-frequencies = /bits/ 64 <456000000>;
+                       };
+               };
+       };
 };

 &pcie_phy {
@@ -297,6 +336,12 @@
                >;
        };

+       pinctrl_imx219: imx219grp {
+               fsl,pins = <
+                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
+               >;
+       };
+
        pinctrl_pcie0: pcie0grp {
                fsl,pins = <
                        MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41

While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
currently supports only 2 lanes, and a few different resolutions
including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
640x480@30fps (cropped) with RAW8 and RAW10 formats.

I'm setting up the pipeline like this:
media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"

and capture:
gst-launch-1.0 -v v4l2src num-buffers=1 !
video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
filesink location=test

The above hangs after:
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
framerate=(fraction)30/1, interlace-mode=(string)progressive
New clock: GstSystemClock
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
framerate=(fraction)30/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
framerate=(fraction)30/1, interlace-mode=(string)progressive
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
framerate=(fraction)30/1, interlace-mode=(string)progressive

I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
for RAW formats' patch with the same results.

Let me know if any of you have some ideas here.

Best regards,

Tim

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

On Mon, Nov 1, 2021 at 6:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
>
> On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
> <frieder.schrempf@kontron.de> wrote:
> >
> > On 28.10.21 02:39, Adam Ford wrote:
> > > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> > >>
> > >> Hi Adam,
> > >>
> > >> [Adding Frieder on Cc]
> > >>
> > >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > >>>
> > >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > >>> use an OV5640 camera.
> > >>>
> > >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > >>>
> > >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > >>> pipeline can be configured with the following:
> > >>>
> > >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > >>>
> > >>> The camera and various nodes in the pipeline can be configured for UYVY:
> > >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > >>>
> > >>> With that, the media pipeline looks like:
> > >>>
> > >>>
> > >>> Media controller API version 5.15.0
> > >>>
> > >>> Media device information
> > >>> ------------------------
> > >>> driver          imx7-csi
> > >>> model           imx-media
> > >>> serial
> > >>> bus info        platform:32e20000.csi
> > >>> hw revision     0x0
> > >>> driver version  5.15.0
> > >>>
> > >>> Device topology
> > >>> - entity 1: csi (2 pads, 2 links)
> > >>>             type V4L2 subdev subtype Unknown flags 0
> > >>>             device node name /dev/v4l-subdev0
> > >>>         pad0: Sink
> > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > >>>         pad1: Source
> > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > >>>
> > >>> - entity 4: csi capture (1 pad, 1 link)
> > >>>             type Node subtype V4L flags 0
> > >>>             device node name /dev/video0
> > >>>         pad0: Sink
> > >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> > >>>
> > >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > >>>              type V4L2 subdev subtype Unknown flags 0
> > >>>              device node name /dev/v4l-subdev1
> > >>>         pad0: Sink
> > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > >>>                 <- "ov5640 1-003c":0 [ENABLED]
> > >>>         pad1: Source
> > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> > >>>
> > >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> > >>>              type V4L2 subdev subtype Sensor flags 0
> > >>>              device node name /dev/v4l-subdev2
> > >>>         pad0: Source
> > >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > >>>
> > >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > >>>
> > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > >>>
> > >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> > >>> interrupts are recorded and no errors are recorded.
> > >>>
> > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > >>> Setting pipeline to PAUSED ...
> > >>> Pipeline is live and does not need PREROLL ...
> > >>> Pipeline is PREROLLED ...
> > >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > >>> PLAYING ...
> > >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > >>> Clock
> > >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > >>>
> > >>>
> > >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> > >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > >>> I do not have this hardware, so I cannot test it.
> > >>
> > >> It seems that Frieder on Cc managed to get camera capture to work on
> > >> i.MX8MM here:
> > >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> > >>
> > >> Hopefully, this can help to figure out what is missing in mainline to
> > >> get camera capture to work on i.MX8M.
> > >>
> > >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > >> board to try your series.
> > >
> > > Fabio,
> > >
> > > Thanks for the heads up on that repo.  I was able to use that repo and
> > > get still images to capture on an OV5640, but I noticed a fair amount
> > > of differences between that repo and what's found in linux-next.
> > >
> > > Laurent,
> > >
> > > I haven't exhausted the patch differences, but I found at least a few
> > > that appear to be missiing upstream, and I am curious to know if/what
> > > your opinion is on whether or not they're needed, since the patches on
> > > Frieder's repo appear to come from you.
> > > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > >
> > > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > >
> > > Maybe these don't need to be applied, but they are 'some' of the
> > > differences that I see between this 5.10 branch and linux-next .  I
> > > know there are more, but
> > >
> > >
> > > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> > >
> > > Frieder et al,
> > >
> > > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > > am curious to see if a regression popped in somewhere, but git bisect
> > > will make this difficult since there is a fair amount of variation
> > > between this custom repo and the upstream.
> >
> > No, I haven't done anything with CSI on a more recent kernel. And I only
> > used CSI with the tree above and the ADV7280M bridge. I don't have any
> > hardware with a sensor/camera.
> >
> > In case you haven't seen this already, here is a thread with some notes
> > about my testing results:
> > https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
> >
>
> For what it's worth I've got another test setup for IMX8MM CSI
> capture. I have a Raspberry Pi Camera module v2 connected to an
> imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
> 4-lane CSI connection.
>
> Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
> the 'blk-ctl series on top of 5.15 and adding support to my dt via:
>
> commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
> Author: Tim Harvey <tharvey@gateworks.com>
> Date:   Wed Oct 27 15:45:23 2021 -0700
>
>     arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2
>
>     Add support for rpi camera module v2 which is an IMX219 8MP module:
>      - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
>      - has its own on-board 24MHz osc so no clock required from baseboard
>      - pin 11 enables 1.8V and 2.8V LDO which is connected to
>        GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
>        support powerdown-gpios and using gpio1 as reset-gpios
>
>     Signed-off-by: Tim Harvey <tharvey@gateworks.com>
>
> diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> index 7b00b6b5bb38..b708c80d884b 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> @@ -35,6 +35,13 @@
>                 };
>         };
>
> +       cam24m: cam24m {
> +               compatible = "fixed-clock";
> +               #clock-cells = <0>;
> +               clock-frequency = <24000000>;
> +               clock-output-names = "cam24m";
> +       };
> +
>         pcie0_refclk: pcie0-refclk {
>                 compatible = "fixed-clock";
>                 #clock-cells = <0>;
> @@ -100,6 +107,19 @@
>         };
>  };
>
> +&csi {
> +       status = "okay";
> +};
> +
> +&imx8mm_mipi_csi_in {
> +       remote-endpoint = <&imx219_to_mipi_csi2>;
> +       data-lanes = <1 2>;
> +};
> +
> +&mipi_csi2 {
> +       status = "okay";
> +};
> +
>  /* off-board header */
>  &ecspi2 {
>         pinctrl-names = "default";
> @@ -132,6 +152,25 @@
>         pinctrl-names = "default";
>         pinctrl-0 = <&pinctrl_i2c3>;
>         status = "okay";
> +
> +       imx219: sensor@10 {
> +               compatible = "sony,imx219";
> +               pinctrl-names = "default";
> +               pinctrl-0 = <&pinctrl_imx219>;
> +               reg = <0x10>;
> +               clocks = <&cam24m>;
> +               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> +
> +               port {
> +                       /* MIPI CSI-2 bus endpoint */
> +                       imx219_to_mipi_csi2: endpoint {
> +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> +                               clock-lanes = <0>;
> +                               data-lanes = <1 2>;
> +                               link-frequencies = /bits/ 64 <456000000>;
> +                       };
> +               };
> +       };
>  };
>
>  &pcie_phy {
> @@ -297,6 +336,12 @@
>                 >;
>         };
>
> +       pinctrl_imx219: imx219grp {
> +               fsl,pins = <
> +                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
> +               >;
> +       };
> +
>         pinctrl_pcie0: pcie0grp {
>                 fsl,pins = <
>                         MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41
>
> While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
> resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
> currently supports only 2 lanes, and a few different resolutions
> including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
> 640x480@30fps (cropped) with RAW8 and RAW10 formats.

I am hoping to support this camera as well once I get the OV5640 working.

>
> I'm setting up the pipeline like this:
> media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
> media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
> media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"
>
> and capture:
> gst-launch-1.0 -v v4l2src num-buffers=1 !
> video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
> filesink location=test
>
> The above hangs after:
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Setting pipeline to PLAYING ...
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
> video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> framerate=(fraction)30/1, interlace-mode=(string)progressive
> New clock: GstSystemClock
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
> video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> framerate=(fraction)30/1, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
> video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> framerate=(fraction)30/1, interlace-mode=(string)progressive
> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
> video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> framerate=(fraction)30/1, interlace-mode=(string)progressive
>
> I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
> for RAW formats' patch with the same results.

Same here.

>
> Let me know if any of you have some ideas here.

Tim,

Can you check /proc/interrupts?  I assume that you've got no interrupts either.

I've added a bunch of debug code to the 5.10 NXP kernel and dumped a
bunch of the regisgters and compared it to the ov5640 running on the
beacon board.  As of now, I think the issue is somewhere in the CSI
Bridge driver.  I've made a few changes to make the CSI bridge
registers match that of the 5.10 NXP kernel, but I haven't found the
magic register yet.  In my setup, I can get the CSIS driver registers
to match the NXP kernel, so I'm leaning toward either the CSI bridge
driver or an order of operations difference between the mainline
kernel and NXP's.

I'm working on this in my spare time, so I'll keep you posted if I
make any progress.

adam
>
> Best regards,
>
> Tim

Re: [RFC V2 5/5] arm64: dts: imx8mm-evk: Enable OV5640 Camera

[Tim Harvey]

On Sat, Oct 23, 2021 at 1:39 PM Adam Ford <aford173@gmail.com> wrote:
>
> The schematic shows support for a camera interface, and the NXP
> kernel shows it is an OV5640.
>
> Signed-off-by: Adam Ford <aford173@gmail.com>
> ---
>  arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi | 44 +++++++++++++++++++
>  1 file changed, 44 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> index e033d0257b5a..27217d30b8d8 100644
> --- a/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> +++ b/arch/arm64/boot/dts/freescale/imx8mm-evk.dtsi
> @@ -239,6 +239,10 @@ ldo6_reg: LDO6 {
>         };
>  };
>
> +&csi {
> +       status = "okay";
> +};
> +
>  &i2c2 {
>         clock-frequency = <400000>;
>         pinctrl-names = "default";
> @@ -287,6 +291,38 @@ pca6416: gpio@20 {
>                 gpio-controller;
>                 #gpio-cells = <2>;
>         };
> +
> +       camera@3c {
> +               compatible = "ovti,ov5640";
> +               pinctrl-names = "default";
> +               pinctrl-0 = <&pinctrl_ov5640>;
> +               reg = <0x3c>;
> +               clocks = <&clk IMX8MM_CLK_CLKO1>;
> +               clock-names = "xclk";
> +               assigned-clocks = <&clk IMX8MM_CLK_CLKO1>;
> +               assigned-clock-parents = <&clk IMX8MM_CLK_24M>;
> +               assigned-clock-rates = <24000000>;
> +               powerdown-gpios = <&gpio1 7 GPIO_ACTIVE_HIGH>;
> +               reset-gpios = <&gpio1 6 GPIO_ACTIVE_LOW>;
> +
> +               port {
> +                       /* MIPI CSI-2 bus endpoint */
> +                       ov5640_to_mipi_csi2: endpoint {
> +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> +                               clock-lanes = <0>;
> +                               data-lanes = <1 2>;
> +                       };
> +               };
> +       };

Adam,

On the imx8mm-evk the ov5640 is on i2c3

Tim

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Tim Harvey]

On Mon, Nov 1, 2021 at 5:30 PM Adam Ford <aford173@gmail.com> wrote:
>
> On Mon, Nov 1, 2021 at 6:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
> >
> > On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
> > <frieder.schrempf@kontron.de> wrote:
> > >
> > > On 28.10.21 02:39, Adam Ford wrote:
> > > > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> > > >>
> > > >> Hi Adam,
> > > >>
> > > >> [Adding Frieder on Cc]
> > > >>
> > > >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > > >>>
> > > >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > > >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > > >>> use an OV5640 camera.
> > > >>>
> > > >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > > >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > > >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > > >>>
> > > >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > > >>> pipeline can be configured with the following:
> > > >>>
> > > >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > > >>>
> > > >>> The camera and various nodes in the pipeline can be configured for UYVY:
> > > >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > >>>
> > > >>> With that, the media pipeline looks like:
> > > >>>
> > > >>>
> > > >>> Media controller API version 5.15.0
> > > >>>
> > > >>> Media device information
> > > >>> ------------------------
> > > >>> driver          imx7-csi
> > > >>> model           imx-media
> > > >>> serial
> > > >>> bus info        platform:32e20000.csi
> > > >>> hw revision     0x0
> > > >>> driver version  5.15.0
> > > >>>
> > > >>> Device topology
> > > >>> - entity 1: csi (2 pads, 2 links)
> > > >>>             type V4L2 subdev subtype Unknown flags 0
> > > >>>             device node name /dev/v4l-subdev0
> > > >>>         pad0: Sink
> > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > > >>>         pad1: Source
> > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > >>>
> > > >>> - entity 4: csi capture (1 pad, 1 link)
> > > >>>             type Node subtype V4L flags 0
> > > >>>             device node name /dev/video0
> > > >>>         pad0: Sink
> > > >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> > > >>>
> > > >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > > >>>              type V4L2 subdev subtype Unknown flags 0
> > > >>>              device node name /dev/v4l-subdev1
> > > >>>         pad0: Sink
> > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > >>>                 <- "ov5640 1-003c":0 [ENABLED]
> > > >>>         pad1: Source
> > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> > > >>>
> > > >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> > > >>>              type V4L2 subdev subtype Sensor flags 0
> > > >>>              device node name /dev/v4l-subdev2
> > > >>>         pad0: Source
> > > >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > > >>>
> > > >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > > >>>
> > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > >>>
> > > >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> > > >>> interrupts are recorded and no errors are recorded.
> > > >>>
> > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > >>> Setting pipeline to PAUSED ...
> > > >>> Pipeline is live and does not need PREROLL ...
> > > >>> Pipeline is PREROLLED ...
> > > >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > > >>> PLAYING ...
> > > >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > > >>> Clock
> > > >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > >>>
> > > >>>
> > > >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> > > >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > > >>> I do not have this hardware, so I cannot test it.
> > > >>
> > > >> It seems that Frieder on Cc managed to get camera capture to work on
> > > >> i.MX8MM here:
> > > >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> > > >>
> > > >> Hopefully, this can help to figure out what is missing in mainline to
> > > >> get camera capture to work on i.MX8M.
> > > >>
> > > >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > > >> board to try your series.
> > > >
> > > > Fabio,
> > > >
> > > > Thanks for the heads up on that repo.  I was able to use that repo and
> > > > get still images to capture on an OV5640, but I noticed a fair amount
> > > > of differences between that repo and what's found in linux-next.
> > > >
> > > > Laurent,
> > > >
> > > > I haven't exhausted the patch differences, but I found at least a few
> > > > that appear to be missiing upstream, and I am curious to know if/what
> > > > your opinion is on whether or not they're needed, since the patches on
> > > > Frieder's repo appear to come from you.
> > > > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > > > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > > > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > >
> > > > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > >
> > > > Maybe these don't need to be applied, but they are 'some' of the
> > > > differences that I see between this 5.10 branch and linux-next .  I
> > > > know there are more, but
> > > >
> > > >
> > > > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > > > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > > > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> > > >
> > > > Frieder et al,
> > > >
> > > > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > > > am curious to see if a regression popped in somewhere, but git bisect
> > > > will make this difficult since there is a fair amount of variation
> > > > between this custom repo and the upstream.
> > >
> > > No, I haven't done anything with CSI on a more recent kernel. And I only
> > > used CSI with the tree above and the ADV7280M bridge. I don't have any
> > > hardware with a sensor/camera.
> > >
> > > In case you haven't seen this already, here is a thread with some notes
> > > about my testing results:
> > > https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
> > >
> >
> > For what it's worth I've got another test setup for IMX8MM CSI
> > capture. I have a Raspberry Pi Camera module v2 connected to an
> > imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
> > 4-lane CSI connection.
> >
> > Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
> > the 'blk-ctl series on top of 5.15 and adding support to my dt via:
> >
> > commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
> > Author: Tim Harvey <tharvey@gateworks.com>
> > Date:   Wed Oct 27 15:45:23 2021 -0700
> >
> >     arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2
> >
> >     Add support for rpi camera module v2 which is an IMX219 8MP module:
> >      - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
> >      - has its own on-board 24MHz osc so no clock required from baseboard
> >      - pin 11 enables 1.8V and 2.8V LDO which is connected to
> >        GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
> >        support powerdown-gpios and using gpio1 as reset-gpios
> >
> >     Signed-off-by: Tim Harvey <tharvey@gateworks.com>
> >
> > diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > index 7b00b6b5bb38..b708c80d884b 100644
> > --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > @@ -35,6 +35,13 @@
> >                 };
> >         };
> >
> > +       cam24m: cam24m {
> > +               compatible = "fixed-clock";
> > +               #clock-cells = <0>;
> > +               clock-frequency = <24000000>;
> > +               clock-output-names = "cam24m";
> > +       };
> > +
> >         pcie0_refclk: pcie0-refclk {
> >                 compatible = "fixed-clock";
> >                 #clock-cells = <0>;
> > @@ -100,6 +107,19 @@
> >         };
> >  };
> >
> > +&csi {
> > +       status = "okay";
> > +};
> > +
> > +&imx8mm_mipi_csi_in {
> > +       remote-endpoint = <&imx219_to_mipi_csi2>;
> > +       data-lanes = <1 2>;
> > +};
> > +
> > +&mipi_csi2 {
> > +       status = "okay";
> > +};
> > +
> >  /* off-board header */
> >  &ecspi2 {
> >         pinctrl-names = "default";
> > @@ -132,6 +152,25 @@
> >         pinctrl-names = "default";
> >         pinctrl-0 = <&pinctrl_i2c3>;
> >         status = "okay";
> > +
> > +       imx219: sensor@10 {
> > +               compatible = "sony,imx219";
> > +               pinctrl-names = "default";
> > +               pinctrl-0 = <&pinctrl_imx219>;
> > +               reg = <0x10>;
> > +               clocks = <&cam24m>;
> > +               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> > +
> > +               port {
> > +                       /* MIPI CSI-2 bus endpoint */
> > +                       imx219_to_mipi_csi2: endpoint {
> > +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> > +                               clock-lanes = <0>;
> > +                               data-lanes = <1 2>;
> > +                               link-frequencies = /bits/ 64 <456000000>;
> > +                       };
> > +               };
> > +       };
> >  };
> >
> >  &pcie_phy {
> > @@ -297,6 +336,12 @@
> >                 >;
> >         };
> >
> > +       pinctrl_imx219: imx219grp {
> > +               fsl,pins = <
> > +                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
> > +               >;
> > +       };
> > +
> >         pinctrl_pcie0: pcie0grp {
> >                 fsl,pins = <
> >                         MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41
> >
> > While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
> > resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
> > currently supports only 2 lanes, and a few different resolutions
> > including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
> > 640x480@30fps (cropped) with RAW8 and RAW10 formats.
>
> I am hoping to support this camera as well once I get the OV5640 working.
>
> >
> > I'm setting up the pipeline like this:
> > media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
> > media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
> > media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"
> >
> > and capture:
> > gst-launch-1.0 -v v4l2src num-buffers=1 !
> > video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
> > filesink location=test
> >
> > The above hangs after:
> > Setting pipeline to PAUSED ...
> > Pipeline is live and does not need PREROLL ...
> > Setting pipeline to PLAYING ...
> > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
> > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > New clock: GstSystemClock
> > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
> > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
> > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
> > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > framerate=(fraction)30/1, interlace-mode=(string)progressive
> >
> > I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
> > for RAW formats' patch with the same results.
>
> Same here.
>
> >
> > Let me know if any of you have some ideas here.
>
> Tim,
>
> Can you check /proc/interrupts?  I assume that you've got no interrupts either.

Adam,

like your setup I see:
 52:          0          0          0          0     GICv3  48 Level     csi
 53:          0          0          0          0     GICv3  49 Level
  32e30000.mipi-csi
Err:          0

clk_summary:
                                 enable  prepare  protect
                  duty  hardware
   clock                          count    count    count        rate
 accuracy phase  cycle    enable
-------------------------------------------------------------------------------------------------------
 sys_pll2                             6        6        0  1000000000
        0     0  50000         Y
    sys_pll2_out                      1        1        0  1000000000
        0     0  50000         Y
       sys_pll2_1000m                 3        3        0  1000000000
        0     0  50000         Y
          csi1_phy_ref                2        2        0  1000000000
        0     0  50000         Y
          csi1_core                   1        1        0   333333334
        0     0  50000         Y
             csi1_root_clk            3        3        0   333333334
        0     0  50000         Y

>
> I've added a bunch of debug code to the 5.10 NXP kernel and dumped a
> bunch of the regisgters and compared it to the ov5640 running on the
> beacon board.  As of now, I think the issue is somewhere in the CSI
> Bridge driver.  I've made a few changes to make the CSI bridge
> registers match that of the 5.10 NXP kernel, but I haven't found the
> magic register yet.  In my setup, I can get the CSIS driver registers
> to match the NXP kernel, so I'm leaning toward either the CSI bridge
> driver or an order of operations difference between the mainline
> kernel and NXP's.
>
> I'm working on this in my spare time, so I'll keep you posted if I
> make any progress.
>

Sounds good, let me know if you need more testing.

I also have an imx8mm-evk here with the ov5640 camera so can do some
testing there as well. I'm not clear how to use the imx8mm-evk with
NXP's lf-5.10.y downstream vendor kernel in order to compare as it
uses their proprietary platform capture driver instead of using the
media controller API and I'm not clear how to set that device up for
capture.

However if I test that board using the 5.15 with your series (adjusted
for the ov5640 camera on i2c3 for the imx8mm-evk) I get the same
results as you do also - no interrupts.

Best regards,

Tim

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

On Tue, Nov 2, 2021 at 12:50 PM Tim Harvey <tharvey@gateworks.com> wrote:
>
> On Mon, Nov 1, 2021 at 5:30 PM Adam Ford <aford173@gmail.com> wrote:
> >
> > On Mon, Nov 1, 2021 at 6:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
> > >
> > > On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
> > > <frieder.schrempf@kontron.de> wrote:
> > > >
> > > > On 28.10.21 02:39, Adam Ford wrote:
> > > > > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> > > > >>
> > > > >> Hi Adam,
> > > > >>
> > > > >> [Adding Frieder on Cc]
> > > > >>
> > > > >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > > > >>>
> > > > >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > > > >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > > > >>> use an OV5640 camera.
> > > > >>>
> > > > >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > > > >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > > > >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > > > >>>
> > > > >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > > > >>> pipeline can be configured with the following:
> > > > >>>
> > > > >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > > > >>>
> > > > >>> The camera and various nodes in the pipeline can be configured for UYVY:
> > > > >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > >>>
> > > > >>> With that, the media pipeline looks like:
> > > > >>>
> > > > >>>
> > > > >>> Media controller API version 5.15.0
> > > > >>>
> > > > >>> Media device information
> > > > >>> ------------------------
> > > > >>> driver          imx7-csi
> > > > >>> model           imx-media
> > > > >>> serial
> > > > >>> bus info        platform:32e20000.csi
> > > > >>> hw revision     0x0
> > > > >>> driver version  5.15.0
> > > > >>>
> > > > >>> Device topology
> > > > >>> - entity 1: csi (2 pads, 2 links)
> > > > >>>             type V4L2 subdev subtype Unknown flags 0
> > > > >>>             device node name /dev/v4l-subdev0
> > > > >>>         pad0: Sink
> > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > > > >>>         pad1: Source
> > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > > >>>
> > > > >>> - entity 4: csi capture (1 pad, 1 link)
> > > > >>>             type Node subtype V4L flags 0
> > > > >>>             device node name /dev/video0
> > > > >>>         pad0: Sink
> > > > >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> > > > >>>
> > > > >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > > > >>>              type V4L2 subdev subtype Unknown flags 0
> > > > >>>              device node name /dev/v4l-subdev1
> > > > >>>         pad0: Sink
> > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > >>>                 <- "ov5640 1-003c":0 [ENABLED]
> > > > >>>         pad1: Source
> > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> > > > >>>
> > > > >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> > > > >>>              type V4L2 subdev subtype Sensor flags 0
> > > > >>>              device node name /dev/v4l-subdev2
> > > > >>>         pad0: Source
> > > > >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > > >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > > > >>>
> > > > >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > > > >>>
> > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > >>>
> > > > >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> > > > >>> interrupts are recorded and no errors are recorded.
> > > > >>>
> > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > >>> Setting pipeline to PAUSED ...
> > > > >>> Pipeline is live and does not need PREROLL ...
> > > > >>> Pipeline is PREROLLED ...
> > > > >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > > > >>> PLAYING ...
> > > > >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > > > >>> Clock
> > > > >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > >>>
> > > > >>>
> > > > >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> > > > >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > > > >>> I do not have this hardware, so I cannot test it.
> > > > >>
> > > > >> It seems that Frieder on Cc managed to get camera capture to work on
> > > > >> i.MX8MM here:
> > > > >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> > > > >>
> > > > >> Hopefully, this can help to figure out what is missing in mainline to
> > > > >> get camera capture to work on i.MX8M.
> > > > >>
> > > > >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > > > >> board to try your series.
> > > > >
> > > > > Fabio,
> > > > >
> > > > > Thanks for the heads up on that repo.  I was able to use that repo and
> > > > > get still images to capture on an OV5640, but I noticed a fair amount
> > > > > of differences between that repo and what's found in linux-next.
> > > > >
> > > > > Laurent,
> > > > >
> > > > > I haven't exhausted the patch differences, but I found at least a few
> > > > > that appear to be missiing upstream, and I am curious to know if/what
> > > > > your opinion is on whether or not they're needed, since the patches on
> > > > > Frieder's repo appear to come from you.
> > > > > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > > > > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > > > > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > >
> > > > > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > >
> > > > > Maybe these don't need to be applied, but they are 'some' of the
> > > > > differences that I see between this 5.10 branch and linux-next .  I
> > > > > know there are more, but
> > > > >
> > > > >
> > > > > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > > > > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > > > > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> > > > >
> > > > > Frieder et al,
> > > > >
> > > > > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > > > > am curious to see if a regression popped in somewhere, but git bisect
> > > > > will make this difficult since there is a fair amount of variation
> > > > > between this custom repo and the upstream.
> > > >
> > > > No, I haven't done anything with CSI on a more recent kernel. And I only
> > > > used CSI with the tree above and the ADV7280M bridge. I don't have any
> > > > hardware with a sensor/camera.
> > > >
> > > > In case you haven't seen this already, here is a thread with some notes
> > > > about my testing results:
> > > > https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
> > > >
> > >
> > > For what it's worth I've got another test setup for IMX8MM CSI
> > > capture. I have a Raspberry Pi Camera module v2 connected to an
> > > imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
> > > 4-lane CSI connection.
> > >
> > > Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
> > > the 'blk-ctl series on top of 5.15 and adding support to my dt via:
> > >
> > > commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
> > > Author: Tim Harvey <tharvey@gateworks.com>
> > > Date:   Wed Oct 27 15:45:23 2021 -0700
> > >
> > >     arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2
> > >
> > >     Add support for rpi camera module v2 which is an IMX219 8MP module:
> > >      - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
> > >      - has its own on-board 24MHz osc so no clock required from baseboard
> > >      - pin 11 enables 1.8V and 2.8V LDO which is connected to
> > >        GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
> > >        support powerdown-gpios and using gpio1 as reset-gpios
> > >
> > >     Signed-off-by: Tim Harvey <tharvey@gateworks.com>
> > >
> > > diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > index 7b00b6b5bb38..b708c80d884b 100644
> > > --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > @@ -35,6 +35,13 @@
> > >                 };
> > >         };
> > >
> > > +       cam24m: cam24m {
> > > +               compatible = "fixed-clock";
> > > +               #clock-cells = <0>;
> > > +               clock-frequency = <24000000>;
> > > +               clock-output-names = "cam24m";
> > > +       };
> > > +
> > >         pcie0_refclk: pcie0-refclk {
> > >                 compatible = "fixed-clock";
> > >                 #clock-cells = <0>;
> > > @@ -100,6 +107,19 @@
> > >         };
> > >  };
> > >
> > > +&csi {
> > > +       status = "okay";
> > > +};
> > > +
> > > +&imx8mm_mipi_csi_in {
> > > +       remote-endpoint = <&imx219_to_mipi_csi2>;
> > > +       data-lanes = <1 2>;
> > > +};
> > > +
> > > +&mipi_csi2 {
> > > +       status = "okay";
> > > +};
> > > +
> > >  /* off-board header */
> > >  &ecspi2 {
> > >         pinctrl-names = "default";
> > > @@ -132,6 +152,25 @@
> > >         pinctrl-names = "default";
> > >         pinctrl-0 = <&pinctrl_i2c3>;
> > >         status = "okay";
> > > +
> > > +       imx219: sensor@10 {
> > > +               compatible = "sony,imx219";
> > > +               pinctrl-names = "default";
> > > +               pinctrl-0 = <&pinctrl_imx219>;
> > > +               reg = <0x10>;
> > > +               clocks = <&cam24m>;
> > > +               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> > > +
> > > +               port {
> > > +                       /* MIPI CSI-2 bus endpoint */
> > > +                       imx219_to_mipi_csi2: endpoint {
> > > +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> > > +                               clock-lanes = <0>;
> > > +                               data-lanes = <1 2>;
> > > +                               link-frequencies = /bits/ 64 <456000000>;
> > > +                       };
> > > +               };
> > > +       };
> > >  };
> > >
> > >  &pcie_phy {
> > > @@ -297,6 +336,12 @@
> > >                 >;
> > >         };
> > >
> > > +       pinctrl_imx219: imx219grp {
> > > +               fsl,pins = <
> > > +                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
> > > +               >;
> > > +       };
> > > +
> > >         pinctrl_pcie0: pcie0grp {
> > >                 fsl,pins = <
> > >                         MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41
> > >
> > > While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
> > > resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
> > > currently supports only 2 lanes, and a few different resolutions
> > > including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
> > > 640x480@30fps (cropped) with RAW8 and RAW10 formats.
> >
> > I am hoping to support this camera as well once I get the OV5640 working.
> >
> > >
> > > I'm setting up the pipeline like this:
> > > media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
> > > media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
> > > media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"
> > >
> > > and capture:
> > > gst-launch-1.0 -v v4l2src num-buffers=1 !
> > > video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
> > > filesink location=test
> > >
> > > The above hangs after:
> > > Setting pipeline to PAUSED ...
> > > Pipeline is live and does not need PREROLL ...
> > > Setting pipeline to PLAYING ...
> > > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
> > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > New clock: GstSystemClock
> > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
> > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
> > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
> > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > >
> > > I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
> > > for RAW formats' patch with the same results.
> >
> > Same here.
> >
> > >
> > > Let me know if any of you have some ideas here.
> >
> > Tim,
> >
> > Can you check /proc/interrupts?  I assume that you've got no interrupts either.
>
> Adam,
>
> like your setup I see:
>  52:          0          0          0          0     GICv3  48 Level     csi
>  53:          0          0          0          0     GICv3  49 Level
>   32e30000.mipi-csi
> Err:          0
>
> clk_summary:
>                                  enable  prepare  protect
>                   duty  hardware
>    clock                          count    count    count        rate
>  accuracy phase  cycle    enable
> -------------------------------------------------------------------------------------------------------
>  sys_pll2                             6        6        0  1000000000
>         0     0  50000         Y
>     sys_pll2_out                      1        1        0  1000000000
>         0     0  50000         Y
>        sys_pll2_1000m                 3        3        0  1000000000
>         0     0  50000         Y
>           csi1_phy_ref                2        2        0  1000000000
>         0     0  50000         Y
>           csi1_core                   1        1        0   333333334
>         0     0  50000         Y
>              csi1_root_clk            3        3        0   333333334
>         0     0  50000         Y
>
> >
> > I've added a bunch of debug code to the 5.10 NXP kernel and dumped a
> > bunch of the regisgters and compared it to the ov5640 running on the
> > beacon board.  As of now, I think the issue is somewhere in the CSI
> > Bridge driver.  I've made a few changes to make the CSI bridge
> > registers match that of the 5.10 NXP kernel, but I haven't found the
> > magic register yet.  In my setup, I can get the CSIS driver registers
> > to match the NXP kernel, so I'm leaning toward either the CSI bridge
> > driver or an order of operations difference between the mainline
> > kernel and NXP's.
> >
> > I'm working on this in my spare time, so I'll keep you posted if I
> > make any progress.
> >
>
> Sounds good, let me know if you need more testing.
>
> I also have an imx8mm-evk here with the ov5640 camera so can do some
> testing there as well. I'm not clear how to use the imx8mm-evk with
> NXP's lf-5.10.y downstream vendor kernel in order to compare as it
> uses their proprietary platform capture driver instead of using the
> media controller API and I'm not clear how to set that device up for
> capture.
>
> However if I test that board using the 5.15 with your series (adjusted
> for the ov5640 camera on i2c3 for the imx8mm-evk) I get the same
> results as you do also - no interrupts.

I have put together a list of registers that are set (and in what
order).  I'll try to clean it up and post them.  I think there are a
few registers being set differently, but I am not a CSI-2 expert, so I
do not necessarily know what the 'right' register setting is.

I also noticed that there is a CSIS driver [1]  that's not in the
staging area that seems to be similar to that of the iMX8M Mini, so
I've been tempted to review those register settings to see if it could
be adapted to fit the imx7 and the 8MM, but I think the area to focus
is the CSI Bridge and not the CSIS. I could be wrong, but my CSIS
registers are nearly identical to a working register set from an older
NXP kernel.

adam

[1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/exynos4-is/mipi-csis.c?h=v5.15
>
> Best regards,
>
> Tim

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

[-- Attachment #1: Type: text/plain, Size: 24402 bytes --]

On Tue, Nov 2, 2021 at 1:08 PM Adam Ford <aford173@gmail.com> wrote:
>
> On Tue, Nov 2, 2021 at 12:50 PM Tim Harvey <tharvey@gateworks.com> wrote:
> >
> > On Mon, Nov 1, 2021 at 5:30 PM Adam Ford <aford173@gmail.com> wrote:
> > >
> > > On Mon, Nov 1, 2021 at 6:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
> > > >
> > > > On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
> > > > <frieder.schrempf@kontron.de> wrote:
> > > > >
> > > > > On 28.10.21 02:39, Adam Ford wrote:
> > > > > > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> > > > > >>
> > > > > >> Hi Adam,
> > > > > >>
> > > > > >> [Adding Frieder on Cc]
> > > > > >>
> > > > > >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > > > > >>>
> > > > > >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > > > > >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > > > > >>> use an OV5640 camera.
> > > > > >>>
> > > > > >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > > > > >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > > > > >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > > > > >>>
> > > > > >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > > > > >>> pipeline can be configured with the following:
> > > > > >>>
> > > > > >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > > > > >>>
> > > > > >>> The camera and various nodes in the pipeline can be configured for UYVY:
> > > > > >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > > >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > > >>>
> > > > > >>> With that, the media pipeline looks like:
> > > > > >>>
> > > > > >>>
> > > > > >>> Media controller API version 5.15.0
> > > > > >>>
> > > > > >>> Media device information
> > > > > >>> ------------------------
> > > > > >>> driver          imx7-csi
> > > > > >>> model           imx-media
> > > > > >>> serial
> > > > > >>> bus info        platform:32e20000.csi
> > > > > >>> hw revision     0x0
> > > > > >>> driver version  5.15.0
> > > > > >>>
> > > > > >>> Device topology
> > > > > >>> - entity 1: csi (2 pads, 2 links)
> > > > > >>>             type V4L2 subdev subtype Unknown flags 0
> > > > > >>>             device node name /dev/v4l-subdev0
> > > > > >>>         pad0: Sink
> > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > > >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > > > > >>>         pad1: Source
> > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > > >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > > > >>>
> > > > > >>> - entity 4: csi capture (1 pad, 1 link)
> > > > > >>>             type Node subtype V4L flags 0
> > > > > >>>             device node name /dev/video0
> > > > > >>>         pad0: Sink
> > > > > >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> > > > > >>>
> > > > > >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > > > > >>>              type V4L2 subdev subtype Unknown flags 0
> > > > > >>>              device node name /dev/v4l-subdev1
> > > > > >>>         pad0: Sink
> > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > > >>>                 <- "ov5640 1-003c":0 [ENABLED]
> > > > > >>>         pad1: Source
> > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > > >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> > > > > >>>
> > > > > >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> > > > > >>>              type V4L2 subdev subtype Sensor flags 0
> > > > > >>>              device node name /dev/v4l-subdev2
> > > > > >>>         pad0: Source
> > > > > >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > > > >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > > > > >>>
> > > > > >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > > > > >>>
> > > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > > >>>
> > > > > >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> > > > > >>> interrupts are recorded and no errors are recorded.
> > > > > >>>
> > > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > > >>> Setting pipeline to PAUSED ...
> > > > > >>> Pipeline is live and does not need PREROLL ...
> > > > > >>> Pipeline is PREROLLED ...
> > > > > >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > > > > >>> PLAYING ...
> > > > > >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > > > > >>> Clock
> > > > > >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > >>>
> > > > > >>>
> > > > > >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> > > > > >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > > > > >>> I do not have this hardware, so I cannot test it.
> > > > > >>
> > > > > >> It seems that Frieder on Cc managed to get camera capture to work on
> > > > > >> i.MX8MM here:
> > > > > >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> > > > > >>
> > > > > >> Hopefully, this can help to figure out what is missing in mainline to
> > > > > >> get camera capture to work on i.MX8M.
> > > > > >>
> > > > > >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > > > > >> board to try your series.
> > > > > >
> > > > > > Fabio,
> > > > > >
> > > > > > Thanks for the heads up on that repo.  I was able to use that repo and
> > > > > > get still images to capture on an OV5640, but I noticed a fair amount
> > > > > > of differences between that repo and what's found in linux-next.
> > > > > >
> > > > > > Laurent,
> > > > > >
> > > > > > I haven't exhausted the patch differences, but I found at least a few
> > > > > > that appear to be missiing upstream, and I am curious to know if/what
> > > > > > your opinion is on whether or not they're needed, since the patches on
> > > > > > Frieder's repo appear to come from you.
> > > > > > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > > > > > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > > > > > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > > >
> > > > > > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > > >
> > > > > > Maybe these don't need to be applied, but they are 'some' of the
> > > > > > differences that I see between this 5.10 branch and linux-next .  I
> > > > > > know there are more, but
> > > > > >
> > > > > >
> > > > > > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > > > > > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > > > > > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> > > > > >
> > > > > > Frieder et al,
> > > > > >
> > > > > > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > > > > > am curious to see if a regression popped in somewhere, but git bisect
> > > > > > will make this difficult since there is a fair amount of variation
> > > > > > between this custom repo and the upstream.
> > > > >
> > > > > No, I haven't done anything with CSI on a more recent kernel. And I only
> > > > > used CSI with the tree above and the ADV7280M bridge. I don't have any
> > > > > hardware with a sensor/camera.
> > > > >
> > > > > In case you haven't seen this already, here is a thread with some notes
> > > > > about my testing results:
> > > > > https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
> > > > >
> > > >
> > > > For what it's worth I've got another test setup for IMX8MM CSI
> > > > capture. I have a Raspberry Pi Camera module v2 connected to an
> > > > imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
> > > > 4-lane CSI connection.
> > > >
> > > > Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
> > > > the 'blk-ctl series on top of 5.15 and adding support to my dt via:
> > > >
> > > > commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
> > > > Author: Tim Harvey <tharvey@gateworks.com>
> > > > Date:   Wed Oct 27 15:45:23 2021 -0700
> > > >
> > > >     arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2
> > > >
> > > >     Add support for rpi camera module v2 which is an IMX219 8MP module:
> > > >      - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
> > > >      - has its own on-board 24MHz osc so no clock required from baseboard
> > > >      - pin 11 enables 1.8V and 2.8V LDO which is connected to
> > > >        GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
> > > >        support powerdown-gpios and using gpio1 as reset-gpios
> > > >
> > > >     Signed-off-by: Tim Harvey <tharvey@gateworks.com>
> > > >
> > > > diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > index 7b00b6b5bb38..b708c80d884b 100644
> > > > --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > @@ -35,6 +35,13 @@
> > > >                 };
> > > >         };
> > > >
> > > > +       cam24m: cam24m {
> > > > +               compatible = "fixed-clock";
> > > > +               #clock-cells = <0>;
> > > > +               clock-frequency = <24000000>;
> > > > +               clock-output-names = "cam24m";
> > > > +       };
> > > > +
> > > >         pcie0_refclk: pcie0-refclk {
> > > >                 compatible = "fixed-clock";
> > > >                 #clock-cells = <0>;
> > > > @@ -100,6 +107,19 @@
> > > >         };
> > > >  };
> > > >
> > > > +&csi {
> > > > +       status = "okay";
> > > > +};
> > > > +
> > > > +&imx8mm_mipi_csi_in {
> > > > +       remote-endpoint = <&imx219_to_mipi_csi2>;
> > > > +       data-lanes = <1 2>;
> > > > +};
> > > > +
> > > > +&mipi_csi2 {
> > > > +       status = "okay";
> > > > +};
> > > > +
> > > >  /* off-board header */
> > > >  &ecspi2 {
> > > >         pinctrl-names = "default";
> > > > @@ -132,6 +152,25 @@
> > > >         pinctrl-names = "default";
> > > >         pinctrl-0 = <&pinctrl_i2c3>;
> > > >         status = "okay";
> > > > +
> > > > +       imx219: sensor@10 {
> > > > +               compatible = "sony,imx219";
> > > > +               pinctrl-names = "default";
> > > > +               pinctrl-0 = <&pinctrl_imx219>;
> > > > +               reg = <0x10>;
> > > > +               clocks = <&cam24m>;
> > > > +               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> > > > +
> > > > +               port {
> > > > +                       /* MIPI CSI-2 bus endpoint */
> > > > +                       imx219_to_mipi_csi2: endpoint {
> > > > +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> > > > +                               clock-lanes = <0>;
> > > > +                               data-lanes = <1 2>;
> > > > +                               link-frequencies = /bits/ 64 <456000000>;
> > > > +                       };
> > > > +               };
> > > > +       };
> > > >  };
> > > >
> > > >  &pcie_phy {
> > > > @@ -297,6 +336,12 @@
> > > >                 >;
> > > >         };
> > > >
> > > > +       pinctrl_imx219: imx219grp {
> > > > +               fsl,pins = <
> > > > +                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
> > > > +               >;
> > > > +       };
> > > > +
> > > >         pinctrl_pcie0: pcie0grp {
> > > >                 fsl,pins = <
> > > >                         MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41
> > > >
> > > > While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
> > > > resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
> > > > currently supports only 2 lanes, and a few different resolutions
> > > > including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
> > > > 640x480@30fps (cropped) with RAW8 and RAW10 formats.
> > >
> > > I am hoping to support this camera as well once I get the OV5640 working.
> > >
> > > >
> > > > I'm setting up the pipeline like this:
> > > > media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
> > > > media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
> > > > media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"
> > > >
> > > > and capture:
> > > > gst-launch-1.0 -v v4l2src num-buffers=1 !
> > > > video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
> > > > filesink location=test
> > > >
> > > > The above hangs after:
> > > > Setting pipeline to PAUSED ...
> > > > Pipeline is live and does not need PREROLL ...
> > > > Setting pipeline to PLAYING ...
> > > > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
> > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > New clock: GstSystemClock
> > > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
> > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
> > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
> > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > >
> > > > I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
> > > > for RAW formats' patch with the same results.
> > >
> > > Same here.
> > >
> > > >
> > > > Let me know if any of you have some ideas here.
> > >
> > > Tim,
> > >
> > > Can you check /proc/interrupts?  I assume that you've got no interrupts either.
> >
> > Adam,
> >
> > like your setup I see:
> >  52:          0          0          0          0     GICv3  48 Level     csi
> >  53:          0          0          0          0     GICv3  49 Level
> >   32e30000.mipi-csi
> > Err:          0
> >
> > clk_summary:
> >                                  enable  prepare  protect
> >                   duty  hardware
> >    clock                          count    count    count        rate
> >  accuracy phase  cycle    enable
> > -------------------------------------------------------------------------------------------------------
> >  sys_pll2                             6        6        0  1000000000
> >         0     0  50000         Y
> >     sys_pll2_out                      1        1        0  1000000000
> >         0     0  50000         Y
> >        sys_pll2_1000m                 3        3        0  1000000000
> >         0     0  50000         Y
> >           csi1_phy_ref                2        2        0  1000000000
> >         0     0  50000         Y
> >           csi1_core                   1        1        0   333333334
> >         0     0  50000         Y
> >              csi1_root_clk            3        3        0   333333334
> >         0     0  50000         Y
> >
> > >
> > > I've added a bunch of debug code to the 5.10 NXP kernel and dumped a
> > > bunch of the regisgters and compared it to the ov5640 running on the
> > > beacon board.  As of now, I think the issue is somewhere in the CSI
> > > Bridge driver.  I've made a few changes to make the CSI bridge
> > > registers match that of the 5.10 NXP kernel, but I haven't found the
> > > magic register yet.  In my setup, I can get the CSIS driver registers
> > > to match the NXP kernel, so I'm leaning toward either the CSI bridge
> > > driver or an order of operations difference between the mainline
> > > kernel and NXP's.
> > >
> > > I'm working on this in my spare time, so I'll keep you posted if I
> > > make any progress.
> > >
> >
> > Sounds good, let me know if you need more testing.
> >
> > I also have an imx8mm-evk here with the ov5640 camera so can do some
> > testing there as well. I'm not clear how to use the imx8mm-evk with
> > NXP's lf-5.10.y downstream vendor kernel in order to compare as it
> > uses their proprietary platform capture driver instead of using the
> > media controller API and I'm not clear how to set that device up for
> > capture.
> >
> > However if I test that board using the 5.15 with your series (adjusted
> > for the ov5640 camera on i2c3 for the imx8mm-evk) I get the same
> > results as you do also - no interrupts.
>
> I have put together a list of registers that are set (and in what
> order).  I'll try to clean it up and post them.  I think there are a
> few registers being set differently, but I am not a CSI-2 expert, so I
> do not necessarily know what the 'right' register setting is.
>
> I also noticed that there is a CSIS driver [1]  that's not in the
> staging area that seems to be similar to that of the iMX8M Mini, so
> I've been tempted to review those register settings to see if it could
> be adapted to fit the imx7 and the 8MM, but I think the area to focus
> is the CSI Bridge and not the CSIS. I could be wrong, but my CSIS
> registers are nearly identical to a working register set from an older
> NXP kernel.
>
> adam
>
> [1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/exynos4-is/mipi-csis.c?h=v5.15
> >

--Deleted device tree mailing list since we're not really discussing
device tree issues at this point.

A few more updates:

I am attaching two files.  One with working registers and one without
for comparison.  I did a manual find-replace for the different
registers, so I hope I didn't mess it up, but if something seems wrong
in the working version, I can re-test.

I am still not operational, but I found a few items between the
working version and non-working version that I want to discuss.

In the CSI-Bridge, the IRQ enable function sets a different bit by
name.  I think I have patched the upstream one to match.  I think the
upstream one wrongly enables BIT_RFF_OR_INT when it should be
BIT_RF_OR_INTEN.  I also noticed that the working version has
BIT_SOF_INTEN enabled.  The IRQ handler doesn't currently check for
the SOF interrupt, so it's likely we don't want to enable that IRQ,
but the custom driver does enable it.

@@ -251,7 +252,8 @@ static void imx7_csi_hw_enable_irq(struct imx7_csi *csi)
 {
        u32 cr1 = imx7_csi_reg_read(csi, CSI_CSICR1);

-       cr1 |= BIT_RFF_OR_INT;
+       cr1 |= BIT_SOF_INTEN;
+       cr1 |= BIT_RF_OR_INTEN;
        cr1 |= BIT_FB1_DMA_DONE_INTEN;
        cr1 |= BIT_FB2_DMA_DONE_INTEN;

@@ -262,7 +264,8 @@ static void imx7_csi_hw_disable_irq(struct imx7_csi *csi)
 {
        u32 cr1 = imx7_csi_reg_read(csi, CSI_CSICR1);


-       cr1 &= ~BIT_RFF_OR_INT;
+       cr1 &= ~BIT_SOF_INTEN;
+       cr1 &= ~BIT_RF_OR_INTEN;
        cr1 &= ~BIT_FB1_DMA_DONE_INTEN;
        cr1 &= ~BIT_FB2_DMA_DONE_INTEN;

In the working kernel, the csi configure enables both bits for
BIT_BASEADDR_SWITCH_EN  and  BIT_BASEADDR_SWITCH_SEL.  I don't know
why we want or don't want these, but these do apear to be selected in
the custom kernel.

@@ -464,7 +467,7 @@ static void imx7_csi_configure(struct imx7_csi *csi)
        } else {
                cr1 |= BIT_SOF_POL | BIT_REDGE | BIT_HSYNC_POL | BIT_FCC;

-               cr18 |= BIT_DATA_FROM_MIPI;
+               cr18 |= BIT_DATA_FROM_MIPI | BIT_BASEADDR_SWITCH_EN |
BIT_BASEADDR_SWITCH_SEL;

                switch (csi->format_mbus[IMX7_CSI_PAD_SINK].code) {

In the CSIS driver, the interrupt enable in the working driver enables
the interrupt with 0xf00fffff, but upstream is ffffffff.  Looking at
the data sheet, both appear to me to be setting reserved values.
The upstream driver also enables MIPI_CSIS_DBG_INTR_MSK which doesn't
appear to be documented in the 8MM.

 static void mipi_csis_enable_interrupts(struct csi_state *state, bool on)
 {
-       mipi_csis_write(state, MIPI_CSIS_INT_MSK, on ? 0xffffffff : 0);
-       mipi_csis_write(state, MIPI_CSIS_DBG_INTR_MSK, on ? 0xffffffff : 0);
+       mipi_csis_write(state, MIPI_CSIS_INT_MSK, on ? 0xf00fffff : 0);
+       /* mipi_csis_write(state, MIPI_CSIS_DBG_INTR_MSK, on ?
0xffffffff : 0); */
 }

Since I am not an expert in these drivers, I am hoping someone who
understands this better might be able to better explain what some of
these registers do.

I'll keep working again when I have more spare time.

adam

> > Best regards,
> >
> > Tim

[-- Attachment #2: 8mm-csi-capture-not-working --]
[-- Type: application/octet-stream, Size: 3398 bytes --]

[   57.529082] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020902
[   57.533123] 32E2_0004 CSI-Bridge (CSI_CR2) = c0000000
[   57.537246] 32E2_0008 CSI-Bridge (CSI_CR3) = 00008000
[   57.541300] 32E2_0048 CSI-Bridge (CSI_CR18) = 3c42d030
[   57.545436] 32E2_0014 CSI-Bridge (CSI_RXCNT) = 00012c00
[   57.549572] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 028001e0
[   57.553885] 32E2_0030 CSI-Bridge (CSI_FBUF_PARA) = 00000000
[   57.558843] 32E2_0028 CSI-Bridge (CSI_DMASA_FB1) = 92000000
[   57.565286] 32E2_002c CSI-Bridge (CSI_DMASA_FB2) = 92100000
[   57.569423] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
[   57.581496] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
[   57.594405] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00004002
[   57.601157] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00004100
[   57.607915] 32E3_0040 MIPI-CSI (MIPI_CSI_ISP_CONFIG_CH0) = 00000078
[   57.614779] 32E3_0044 MIPI-CSI (MIPI_CSI_ISP_RESOLUTION_CH0) = 01e00280
[   57.621649] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = 0d800000
[   57.628519] 32E3_0024 MIPI-CSI (MIPI_CSI_ISP_SYNC_CH0) =  00000000
[   57.635383] 32E3_0008 MIPI-CSI (MIPI_CSI_CSIS_CLOCK_CTRL) = 000f0001
[   57.639322] 32E3_0030 MIPI-CSI (MIPI_CSI_DPHY_MASTER_SLAVE_CTRL_LOW) = 000001f4
[   57.643353] 32E3_0034 MIPI-CSI (MIPI_CSI_DPHY_MASTER_SLAVE_CTRL_HIGH) = 00000000
[   57.647386] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00014104
[   57.654167] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00004105
[   57.660946] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = 0d800007
[   57.664972] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = f00fffff
[   57.669003] ov5640_set_stream_mipi 1 # OV5640 Start Streaming
[   57.673208] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020802
[   57.679484] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020822
[   57.686366] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020902
[   57.693213] 32E2_0008 CSI-Bridge (CSI_CR3) = 0000c000
[   57.703092] 32E2_0018 CSI-Bridge (CSI_SR) = 80004000
[   57.709728] 32E2_0000 CSI-Bridge (CSI_CR1) = 011b0902
[   57.716593] 32E2_0008 CSI-Bridge (CSI_CR3) = 000010a0
[   57.723457] 32E2_0048 CSI-Bridge (CSI_CR18) =bc42d030
[   64.419484] csi: wait last EOF timeout   # CONTROL-C Pressed Here
[   64.423275] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020902
[   64.427317] 32E2_0008 CSI-Bridge (CSI_CR3) = 00000020
[   64.431357] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020902
[   64.435393] 32E2_0048 CSI-Bridge (CSI_CR18) = 3c42d030
[   64.439516] ov5640_set_stream_mipi 0   # OV5640 Stop Streaming
[   64.445861] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 00000000
[   64.449893] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00004104
[   64.453844] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = 0d800000
[   64.457914] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 00000000
[   64.461952] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 00004104
[   64.462193] 32E2_0000 CSI-Bridge (CSI_CR1) = 00020912
[   64.465899] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL)  = 0d800000
[   64.473942] 32E2_0004 CSI-Bridge (CSI_CR2) = 00000000
[   64.477974] 32E2_0008 CSI-Bridge (CSI_CR3) = 00008000
[   64.482007] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 03200258
[   64.486139] 32E2_0008 CSI-Bridge (CSI_CR3) = 00004000
[   64.490172] 32E2_0008 CSI-Bridge (CSI_CR3) = 00000000



[-- Attachment #3: 8mm-csi-capture-working --]
[-- Type: application/octet-stream, Size: 6569 bytes --]

[  100.437342] 32E2_0008 CSI-Bridge (CSI_CR3) = 8000
[  100.440489] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  100.443947] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  100.446763] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  100.449555] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  100.452717] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  100.456135] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  100.459220] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  100.462081] 32E2_0008 CSI-Bridge (CSI_CR3) = 8000
[  100.465182] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  100.468602] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  100.471411] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  100.474203] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  100.477379] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  100.480802] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  100.483873] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  100.486681] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 0
[  100.486684] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4000
[  100.490123] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = 0
[  102.851970] 32E2_0008 CSI-Bridge (CSI_CR3) = 8000
[  102.858460] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  102.861915] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  102.864753] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.867604] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  102.870778] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  102.874179] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  102.877256] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.880154] 32E2_0008 CSI-Bridge (CSI_CR3) = 8000
[  102.883234] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  102.886634] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  102.889446] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.892261] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  102.895452] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  102.898877] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  102.901931] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.904750] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 0
[  102.904753] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4000
[  102.908172] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = 0
[  102.918771] 32E2_0008 CSI-Bridge (CSI_CR3) = 8000
[  102.925312] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  102.928734] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  102.931556] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.934348] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  102.937508] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  102.940961] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  102.944036] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  102.949367] 32E2_0004 CSI-Bridge (CSI_CR2) = 2d000
[  102.952658] 32E2_0030 CSI-Bridge (CSI_FBUF_PARA) = 0
[  102.955597] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28001e0
[  102.959019] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  102.962072] 32E2_0000 CSI-Bridge (CSI_CR1) = 20902
[  102.965229] 32E2_0004 CSI-Bridge (CSI_CR2) = 3c42d000
[  103.620181] 32E2_0028 CSI-Bridge (CSI_DMASA_FB1) = 81a00000
[  103.623681] 32E2_002c CSI-Bridge (CSI_DMASA_FB2) = 81b00000
[  103.627333] 32E2_0004 CSI-Bridge (CSI_CR2) = 3c42d000
[  103.630862] 32E2_0000 CSI-Bridge (CSI_CR1) = 20802
[  103.634013] 32E2_0000 CSI-Bridge (CSI_CR1) = 20822
[  103.637188] 32E2_0008 CSI-Bridge (CSI_CR3) = c000
[  103.650653] 32E2_0000 CSI-Bridge (CSI_CR1) = 20902
[  103.653796] 32E2_0018 CSI-Bridge (CSI_SR) = 80004000
[  103.657414] 32E2_0004 CSI-Bridge (CSI_CR2) = bc4ad030
[  103.660952] 32E2_0008 CSI-Bridge (CSI_CR3) = 10a0
[  103.664042] 32E2_0004 CSI-Bridge (CSI_CR2) = c0000000
[  103.667477] 32E2_0000 CSI-Bridge (CSI_CR1) = 11b0902
[  103.670932] 32E2_0004 CSI-Bridge (CSI_CR2) = bc4ad030
[  103.674427] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4002
[  103.674440] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4100
[  103.678039] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL)0 = 878
[  103.681633] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL)4 = 1e00280
[  103.685245] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = d800000
[  103.689210] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL)0 = 78
[  103.693149] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL)8 = 0
[  103.696678] 32E3_0008 MIPI-CSI (MIPI_CSI_CSIS_CLOCK_CTRL) = f0001
[  103.700097] 32E3_0030 MIPI-CSI (MIPI_CSI_DPHY_MASTER_SLAVE_CTRL_LOW) = 1f4
[  103.703778] 32E3_0034 MIPI-CSI (MIPI_CSI_DPHY_MASTER_SLAVE_CTRL_HIGH) = 0
[  103.707368] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 14104
[  103.710786] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4105
[  103.714447] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = d800007
[  103.718056] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = f00fffff
[  103.722003] ov5640_mipi 1-0010: s_stream: 1
[  103.757791] 32E2_0018 CSI-Bridge (CSI_SR) = 80014000
[  103.794348] 32E2_0018 CSI-Bridge (CSI_SR) = 80014000
[  103.827214] 32E2_0018 CSI-Bridge (CSI_SR) = 8a2a4000
[  103.830715] 32E2_0028 CSI-Bridge (CSI_DMASA_FB1) = 81c00000
[  103.834232] 32E2_0018 CSI-Bridge (CSI_SR) = 84214000
[  103.846172] 32E2_0008 CSI-Bridge (CSI_CR3) = 20020
[  103.849365] 32E2_0000 CSI-Bridge (CSI_CR1) = 20902
[  103.849454] 32E2_0018 CSI-Bridge (CSI_SR) = 8b264001
[  103.852554] 32E2_0028 CSI-Bridge (CSI_DMASA_FB1) = 0
[  103.855996] mx6s-csi 32e20000.csi1_bridge: mx6s_csi_irq_handler Rx fifo overflow
[  103.855999] 32E2_0004 CSI-Bridge (CSI_CR2) = 3c4ad030
[  103.856004] 32E2_0000 CSI-Bridge (CSI_CR1) = 20802
[  103.856007] 32E2_0000 CSI-Bridge (CSI_CR1) = 20822
[  103.856010] 32E2_0000 CSI-Bridge (CSI_CR1) = 20902
[  103.856016] 32E2_0008 CSI-Bridge (CSI_CR3) = 24020
[  103.856021] 32E2_0004 CSI-Bridge (CSI_CR2) = bc4ad030
[  103.858938] 32E2_002c CSI-Bridge (CSI_DMASA_FB2) = 0
[  103.888780] 32E2_0030 CSI-Bridge (CSI_FBUF_PARA) = 0
[  103.891678] 32E2_0004 CSI-Bridge (CSI_CR2) = 3c4ad030
[  103.895240] ov5640_mipi 1-0010: s_stream: 0
[  103.900113] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 0
[  103.900118] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4104
[  103.903552] 32E3_0024 MIPI-CSI (MIPI_CSI_DPHY_COMMON_CTRL) = d800000
[  103.908058] 32E2_0008 CSI-Bridge (CSI_CR3) = 28020
[  103.915221] 32E2_0000 CSI-Bridge (CSI_CR1) = 40000800
[  103.918622] 32E2_0004 CSI-Bridge (CSI_CR2) = 0
[  103.921437] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  103.924261] 32E2_0000 CSI-Bridge (CSI_CR1) = 21b12
[  103.927419] 32E2_0034 CSI-Bridge (CSI_IMAG_PARA) = 28003c0
[  103.930840] 32E2_0008 CSI-Bridge (CSI_CR3) = 4000
[  103.933892] 32E2_0008 CSI-Bridge (CSI_CR3) = 0
[  103.936730] 32E3_0010 MIPI-CSI (MIPI_CSI_INTERRUPT_MASK_0) = 0
[  103.936734] 32E3_0004 MIPI-CSI (MIPI_CSI_CSIS_COMMON_CTRL) = 4104

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Adam Ford]

)

On Tue, Nov 2, 2021 at 7:45 PM Adam Ford <aford173@gmail.com> wrote:
>
> On Tue, Nov 2, 2021 at 1:08 PM Adam Ford <aford173@gmail.com> wrote:
> >
> > On Tue, Nov 2, 2021 at 12:50 PM Tim Harvey <tharvey@gateworks.com> wrote:
> > >
> > > On Mon, Nov 1, 2021 at 5:30 PM Adam Ford <aford173@gmail.com> wrote:
> > > >
> > > > On Mon, Nov 1, 2021 at 6:05 PM Tim Harvey <tharvey@gateworks.com> wrote:
> > > > >
> > > > > On Fri, Oct 29, 2021 at 4:11 AM Frieder Schrempf
> > > > > <frieder.schrempf@kontron.de> wrote:
> > > > > >
> > > > > > On 28.10.21 02:39, Adam Ford wrote:
> > > > > > > On Sun, Oct 24, 2021 at 7:16 AM Fabio Estevam <festevam@gmail.com> wrote:
> > > > > > >>
> > > > > > >> Hi Adam,
> > > > > > >>
> > > > > > >> [Adding Frieder on Cc]
> > > > > > >>
> > > > > > >> On Sat, Oct 23, 2021 at 5:35 PM Adam Ford <aford173@gmail.com> wrote:
> > > > > > >>>
> > > > > > >>> The imx8mm appears to have both a CSI bridge and mipi-csi-2 drivers.  With
> > > > > > >>> those enabled, both the imx8mm-evk and imx8mm-beacon boards should be able
> > > > > > >>> use an OV5640 camera.
> > > > > > >>>
> > > > > > >>> The mipi-csi2 driver sets the clock frequency to 333MHz, so the clock parent
> > > > > > >>> of the CSI1 must be reparented to a faster clock.  On the custom NXP kernel,
> > > > > > >>> they use IMX8MM_SYS_PLL2_1000M, so that is done in the device tree to match.
> > > > > > >>>
> > > > > > >>> With the CSI and mipi_csi2 drivers pointing to an OV5640 camera, the media
> > > > > > >>> pipeline can be configured with the following:
> > > > > > >>>
> > > > > > >>>     media-ctl --links "'ov5640 1-003c':0->'imx7-mipi-csis.0':0[1]"
> > > > > > >>>
> > > > > > >>> The camera and various nodes in the pipeline can be configured for UYVY:
> > > > > > >>>     media-ctl -v -V "'ov5640 1-003c':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > > > >>>     media-ctl -v -V "'csi':0 [fmt:UYVY8_1X16/640x480 field:none]"
> > > > > > >>>
> > > > > > >>> With that, the media pipeline looks like:
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Media controller API version 5.15.0
> > > > > > >>>
> > > > > > >>> Media device information
> > > > > > >>> ------------------------
> > > > > > >>> driver          imx7-csi
> > > > > > >>> model           imx-media
> > > > > > >>> serial
> > > > > > >>> bus info        platform:32e20000.csi
> > > > > > >>> hw revision     0x0
> > > > > > >>> driver version  5.15.0
> > > > > > >>>
> > > > > > >>> Device topology
> > > > > > >>> - entity 1: csi (2 pads, 2 links)
> > > > > > >>>             type V4L2 subdev subtype Unknown flags 0
> > > > > > >>>             device node name /dev/v4l-subdev0
> > > > > > >>>         pad0: Sink
> > > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > > > >>>                 <- "imx7-mipi-csis.0":1 [ENABLED,IMMUTABLE]
> > > > > > >>>         pad1: Source
> > > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:lim-range]
> > > > > > >>>                 -> "csi capture":0 [ENABLED,IMMUTABLE]
> > > > > > >>>
> > > > > > >>> - entity 4: csi capture (1 pad, 1 link)
> > > > > > >>>             type Node subtype V4L flags 0
> > > > > > >>>             device node name /dev/video0
> > > > > > >>>         pad0: Sink
> > > > > > >>>                 <- "csi":1 [ENABLED,IMMUTABLE]
> > > > > > >>>
> > > > > > >>> - entity 10: imx7-mipi-csis.0 (2 pads, 2 links)
> > > > > > >>>              type V4L2 subdev subtype Unknown flags 0
> > > > > > >>>              device node name /dev/v4l-subdev1
> > > > > > >>>         pad0: Sink
> > > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > > > >>>                 <- "ov5640 1-003c":0 [ENABLED]
> > > > > > >>>         pad1: Source
> > > > > > >>>                 [fmt:UYVY8_1X16/640x480 field:none colorspace:smpte170m xfer:709 ycbcr:601 quantization:lim-range]
> > > > > > >>>                 -> "csi":0 [ENABLED,IMMUTABLE]
> > > > > > >>>
> > > > > > >>> - entity 15: ov5640 1-003c (1 pad, 1 link)
> > > > > > >>>              type V4L2 subdev subtype Sensor flags 0
> > > > > > >>>              device node name /dev/v4l-subdev2
> > > > > > >>>         pad0: Source
> > > > > > >>>                 [fmt:UYVY8_1X16/640x480@1/30 field:none colorspace:srgb xfer:srgb ycbcr:601 quantization:full-range]
> > > > > > >>>                 -> "imx7-mipi-csis.0":0 [ENABLED]
> > > > > > >>>
> > > > > > >>> When configured, gstreamer can be used to capture 1 frame and store it to a file.
> > > > > > >>>
> > > > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > > > >>>
> > > > > > >>> Unfortunately, the video capture never appears to happen.  No errors occur, not
> > > > > > >>> interrupts are recorded and no errors are recorded.
> > > > > > >>>
> > > > > > >>> gst-launch-1.0 -v v4l2src num-buffers=1 ! video/x-raw,format=UYVY,width=640,height=480,framerate=60/1 ! filesink location=test
> > > > > > >>> Setting pipeline to PAUSED ...
> > > > > > >>> Pipeline is live and does not need PREROLL ...
> > > > > > >>> Pipeline is PREROLLED ...
> > > > > > >>> Setting pipeline to [  114.819632] v4l2_get_link_freq: Link frequency estimated using pixel rate: result might be inaccurate
> > > > > > >>> PLAYING ...
> > > > > > >>> New clock: GstSystem[  114.829203] v4l2_get_link_freq: Consider implementing support for V4L2_CID_LINK_FREQ in the transmitter driver
> > > > > > >>> Clock
> > > > > > >>> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > > >>> /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > > >>> /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw, format=(string)UYVY, width=(int)640, height=(int)480, framerate=(fraction)60/1, interlace-mode=(string)progressive, colorimetry=(string)bt709
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> If anyone has any insight as to what might be wrong, I'd like feedback.
> > > > > > >>> I posted a device tree that I beleive goes with the newer imx8mm-evk, but
> > > > > > >>> I do not have this hardware, so I cannot test it.
> > > > > > >>
> > > > > > >> It seems that Frieder on Cc managed to get camera capture to work on
> > > > > > >> i.MX8MM here:
> > > > > > >> https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommits%2Fv5.10-mx8mm-csi&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=PbGqhzb2mbUA2SD44%2BosK8rNkK12m1LRd6W4tvkawno%3D&amp;reserved=0
> > > > > > >>
> > > > > > >> Hopefully, this can help to figure out what is missing in mainline to
> > > > > > >> get camera capture to work on i.MX8M.
> > > > > > >>
> > > > > > >> I don't have access to an OV5640 camera to connect to the imx8mm-evk
> > > > > > >> board to try your series.
> > > > > > >
> > > > > > > Fabio,
> > > > > > >
> > > > > > > Thanks for the heads up on that repo.  I was able to use that repo and
> > > > > > > get still images to capture on an OV5640, but I noticed a fair amount
> > > > > > > of differences between that repo and what's found in linux-next.
> > > > > > >
> > > > > > > Laurent,
> > > > > > >
> > > > > > > I haven't exhausted the patch differences, but I found at least a few
> > > > > > > that appear to be missiing upstream, and I am curious to know if/what
> > > > > > > your opinion is on whether or not they're needed, since the patches on
> > > > > > > Frieder's repo appear to come from you.
> > > > > > > [1] - media: imx: imx7-media-csi: Add i.MX8MM identification
> > > > > > > [2] - media: imx: imx7_mipi_csis: Don't set reserved CLK_CTRL field on i.MX8MM
> > > > > > > [3] - media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > > > >
> > > > > > > media: imx: imx7_mipi_csis: Set dual pixel mode for RAW formats
> > > > > > >
> > > > > > > Maybe these don't need to be applied, but they are 'some' of the
> > > > > > > differences that I see between this 5.10 branch and linux-next .  I
> > > > > > > know there are more, but
> > > > > > >
> > > > > > >
> > > > > > > [1] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F8ac7ec6db0c260a871038721886dbdb6660ed84c&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979195945%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=j1iuXWljDd8wA5M44KwLCb%2F21tpdOnKZuJazl25bXbQ%3D&amp;reserved=0
> > > > > > > [2] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F0b5727c8eba8c370f7db5eace0243f78992a4dbb&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=buWbZF0tYfVmibQgBbKJM1PF%2Fw7%2BVO9jhXRCI1zf7TI%3D&amp;reserved=0
> > > > > > > [3] - https://eur04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kontron-electronics.de%2Fsw%2Fmisc%2Flinux%2F-%2Fcommit%2F14befa6bc146b10092a6ac5d0ed4d42c87c6cf27&amp;data=04%7C01%7Cfrieder.schrempf%40kontron.de%7Cfe4f7347385f4185b1c608d999ab75b5%7C8c9d3c973fd941c8a2b1646f3942daf1%7C0%7C0%7C637709783979205943%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=60iLhs0G0FtQegNp9XtVxAhvZEcltdAGGMNAm2l1cSs%3D&amp;reserved=0
> > > > > > >
> > > > > > > Frieder et al,
> > > > > > >
> > > > > > > Have you (or anyone) tried CSI cameras on anything newer than 5.10?  I
> > > > > > > am curious to see if a regression popped in somewhere, but git bisect
> > > > > > > will make this difficult since there is a fair amount of variation
> > > > > > > between this custom repo and the upstream.
> > > > > >
> > > > > > No, I haven't done anything with CSI on a more recent kernel. And I only
> > > > > > used CSI with the tree above and the ADV7280M bridge. I don't have any
> > > > > > hardware with a sensor/camera.
> > > > > >
> > > > > > In case you haven't seen this already, here is a thread with some notes
> > > > > > about my testing results:
> > > > > > https://patchwork.kernel.org/project/linux-media/cover/20210215042741.28850-1-laurent.pinchart@ideasonboard.com/.
> > > > > >
> > > > >
> > > > > For what it's worth I've got another test setup for IMX8MM CSI
> > > > > capture. I have a Raspberry Pi Camera module v2 connected to an
> > > > > imx8mm-venice-gw73xx board. This is a IMX219 8.28MP camera with a
> > > > > 4-lane CSI connection.
> > > > >
> > > > > Putting Adam's patch 'arm64: dts: imx8mm: Add CSI nodes' as well as
> > > > > the 'blk-ctl series on top of 5.15 and adding support to my dt via:
> > > > >
> > > > > commit 87f908a57f48bd7375113991434c2923d65506ac (HEAD -> v5.15-venice)
> > > > > Author: Tim Harvey <tharvey@gateworks.com>
> > > > > Date:   Wed Oct 27 15:45:23 2021 -0700
> > > > >
> > > > >     arm64: dts: imx8mm-venice-gw73xx: add rpi camera module v2
> > > > >
> > > > >     Add support for rpi camera module v2 which is an IMX219 8MP module:
> > > > >      - https://datasheets.raspberrypi.com/camera/camera-v2-schematics.pdf
> > > > >      - has its own on-board 24MHz osc so no clock required from baseboard
> > > > >      - pin 11 enables 1.8V and 2.8V LDO which is connected to
> > > > >        GW73xx MIPI_GPIO4 (IMX8MM GPIO1_IO1). imx219 driver does not
> > > > >        support powerdown-gpios and using gpio1 as reset-gpios
> > > > >
> > > > >     Signed-off-by: Tim Harvey <tharvey@gateworks.com>
> > > > >
> > > > > diff --git a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > > b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > > index 7b00b6b5bb38..b708c80d884b 100644
> > > > > --- a/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > > +++ b/arch/arm64/boot/dts/freescale/imx8mm-venice-gw73xx.dtsi
> > > > > @@ -35,6 +35,13 @@
> > > > >                 };
> > > > >         };
> > > > >
> > > > > +       cam24m: cam24m {
> > > > > +               compatible = "fixed-clock";
> > > > > +               #clock-cells = <0>;
> > > > > +               clock-frequency = <24000000>;
> > > > > +               clock-output-names = "cam24m";
> > > > > +       };
> > > > > +
> > > > >         pcie0_refclk: pcie0-refclk {
> > > > >                 compatible = "fixed-clock";
> > > > >                 #clock-cells = <0>;
> > > > > @@ -100,6 +107,19 @@
> > > > >         };
> > > > >  };
> > > > >
> > > > > +&csi {
> > > > > +       status = "okay";
> > > > > +};
> > > > > +
> > > > > +&imx8mm_mipi_csi_in {
> > > > > +       remote-endpoint = <&imx219_to_mipi_csi2>;
> > > > > +       data-lanes = <1 2>;
> > > > > +};
> > > > > +
> > > > > +&mipi_csi2 {
> > > > > +       status = "okay";
> > > > > +};
> > > > > +
> > > > >  /* off-board header */
> > > > >  &ecspi2 {
> > > > >         pinctrl-names = "default";
> > > > > @@ -132,6 +152,25 @@
> > > > >         pinctrl-names = "default";
> > > > >         pinctrl-0 = <&pinctrl_i2c3>;
> > > > >         status = "okay";
> > > > > +
> > > > > +       imx219: sensor@10 {
> > > > > +               compatible = "sony,imx219";
> > > > > +               pinctrl-names = "default";
> > > > > +               pinctrl-0 = <&pinctrl_imx219>;
> > > > > +               reg = <0x10>;
> > > > > +               clocks = <&cam24m>;
> > > > > +               reset-gpios = <&gpio1 1 GPIO_ACTIVE_HIGH>;
> > > > > +
> > > > > +               port {
> > > > > +                       /* MIPI CSI-2 bus endpoint */
> > > > > +                       imx219_to_mipi_csi2: endpoint {
> > > > > +                               remote-endpoint = <&imx8mm_mipi_csi_in>;
> > > > > +                               clock-lanes = <0>;
> > > > > +                               data-lanes = <1 2>;
> > > > > +                               link-frequencies = /bits/ 64 <456000000>;
> > > > > +                       };
> > > > > +               };
> > > > > +       };
> > > > >  };
> > > > >
> > > > >  &pcie_phy {
> > > > > @@ -297,6 +336,12 @@
> > > > >                 >;
> > > > >         };
> > > > >
> > > > > +       pinctrl_imx219: imx219grp {
> > > > > +               fsl,pins = <
> > > > > +                       MX8MM_IOMUXC_GPIO1_IO01_GPIO1_IO1       0x41
> > > > > +               >;
> > > > > +       };
> > > > > +
> > > > >         pinctrl_pcie0: pcie0grp {
> > > > >                 fsl,pins = <
> > > > >                         MX8MM_IOMUXC_SAI1_RXD4_GPIO4_IO6        0x41
> > > > >
> > > > > While the IMX219 supports up to 4 MIPI CSI2 lanes and a variety of
> > > > > resolutions up to 8MP, the IMX219 driver (drivers/media/i2c/imx219.c)
> > > > > currently supports only 2 lanes, and a few different resolutions
> > > > > including 1080p@30fps (cropped FOV), 1640x1232@30fps (2x2 binned),
> > > > > 640x480@30fps (cropped) with RAW8 and RAW10 formats.
> > > >
> > > > I am hoping to support this camera as well once I get the OV5640 working.
> > > >
> > > > >
> > > > > I'm setting up the pipeline like this:
> > > > > media-ctl --links "'imx219 2-0010':0->'imx7-mipi-csis.0':0[1]"
> > > > > media-ctl -v -V "'imx219 2-0010':0 [fmt:SRGGB10/640x480 field:none]"
> > > > > media-ctl -v -V "'csi':0 [fmt:SRGGB10/640x480 field:none]"
> > > > >
> > > > > and capture:
> > > > > gst-launch-1.0 -v v4l2src num-buffers=1 !
> > > > > video/x-bayer,format=rggb,width=640,height=480,framerate=30/1 !
> > > > > filesink location=test
> > > > >
> > > > > The above hangs after:
> > > > > Setting pipeline to PAUSED ...
> > > > > Pipeline is live and does not need PREROLL ...
> > > > > Setting pipeline to PLAYING ...
> > > > > /GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps =
> > > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > > New clock: GstSystemClock
> > > > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps =
> > > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > > /GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps =
> > > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > > /GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps =
> > > > > video/x-bayer, format=(string)rggb, width=(int)640, height=(int)480,
> > > > > framerate=(fraction)30/1, interlace-mode=(string)progressive
> > > > >
> > > > > I've tried Laurent's 'media: imx: imx7_mipi_csis: Set dual pixel mode
> > > > > for RAW formats' patch with the same results.
> > > >
> > > > Same here.
> > > >
> > > > >
> > > > > Let me know if any of you have some ideas here.
> > > >
> > > > Tim,
> > > >
> > > > Can you check /proc/interrupts?  I assume that you've got no interrupts either.
> > >
> > > Adam,
> > >
> > > like your setup I see:
> > >  52:          0          0          0          0     GICv3  48 Level     csi
> > >  53:          0          0          0          0     GICv3  49 Level
> > >   32e30000.mipi-csi
> > > Err:          0
> > >
> > > clk_summary:
> > >                                  enable  prepare  protect
> > >                   duty  hardware
> > >    clock                          count    count    count        rate
> > >  accuracy phase  cycle    enable
> > > -------------------------------------------------------------------------------------------------------
> > >  sys_pll2                             6        6        0  1000000000
> > >         0     0  50000         Y
> > >     sys_pll2_out                      1        1        0  1000000000
> > >         0     0  50000         Y
> > >        sys_pll2_1000m                 3        3        0  1000000000
> > >         0     0  50000         Y
> > >           csi1_phy_ref                2        2        0  1000000000
> > >         0     0  50000         Y
> > >           csi1_core                   1        1        0   333333334
> > >         0     0  50000         Y
> > >              csi1_root_clk            3        3        0   333333334
> > >         0     0  50000         Y
> > >
> > > >
> > > > I've added a bunch of debug code to the 5.10 NXP kernel and dumped a
> > > > bunch of the regisgters and compared it to the ov5640 running on the
> > > > beacon board.  As of now, I think the issue is somewhere in the CSI
> > > > Bridge driver.  I've made a few changes to make the CSI bridge
> > > > registers match that of the 5.10 NXP kernel, but I haven't found the
> > > > magic register yet.  In my setup, I can get the CSIS driver registers
> > > > to match the NXP kernel, so I'm leaning toward either the CSI bridge
> > > > driver or an order of operations difference between the mainline
> > > > kernel and NXP's.
> > > >
> > > > I'm working on this in my spare time, so I'll keep you posted if I
> > > > make any progress.
> > > >
> > >
> > > Sounds good, let me know if you need more testing.
> > >
> > > I also have an imx8mm-evk here with the ov5640 camera so can do some
> > > testing there as well. I'm not clear how to use the imx8mm-evk with
> > > NXP's lf-5.10.y downstream vendor kernel in order to compare as it
> > > uses their proprietary platform capture driver instead of using the
> > > media controller API and I'm not clear how to set that device up for
> > > capture.
> > >
> > > However if I test that board using the 5.15 with your series (adjusted
> > > for the ov5640 camera on i2c3 for the imx8mm-evk) I get the same
> > > results as you do also - no interrupts.
> >
> > I have put together a list of registers that are set (and in what
> > order).  I'll try to clean it up and post them.  I think there are a
> > few registers being set differently, but I am not a CSI-2 expert, so I
> > do not necessarily know what the 'right' register setting is.
> >
> > I also noticed that there is a CSIS driver [1]  that's not in the
> > staging area that seems to be similar to that of the iMX8M Mini, so
> > I've been tempted to review those register settings to see if it could
> > be adapted to fit the imx7 and the 8MM, but I think the area to focus
> > is the CSI Bridge and not the CSIS. I could be wrong, but my CSIS
> > registers are nearly identical to a working register set from an older
> > NXP kernel.
> >
> > adam
> >
> > [1] - https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/media/platform/exynos4-is/mipi-csis.c?h=v5.15
> > >
>
> --Deleted device tree mailing list since we're not really discussing
> device tree issues at this point.
>
> A few more updates:
>
> I am attaching two files.  One with working registers and one without
> for comparison.  I did a manual find-replace for the different
> registers, so I hope I didn't mess it up, but if something seems wrong
> in the working version, I can re-test.
>
> I am still not operational, but I found a few items between the
> working version and non-working version that I want to discuss.
>
> In the CSI-Bridge, the IRQ enable function sets a different bit by
> name.  I think I have patched the upstream one to match.  I think the
> upstream one wrongly enables BIT_RFF_OR_INT when it should be
> BIT_RF_OR_INTEN.  I also noticed that the working version has
> BIT_SOF_INTEN enabled.  The IRQ handler doesn't currently check for
> the SOF interrupt, so it's likely we don't want to enable that IRQ,
> but the custom driver does enable it.
>
> @@ -251,7 +252,8 @@ static void imx7_csi_hw_enable_irq(struct imx7_csi *csi)
>  {
>         u32 cr1 = imx7_csi_reg_read(csi, CSI_CSICR1);
>
> -       cr1 |= BIT_RFF_OR_INT;
> +       cr1 |= BIT_SOF_INTEN;
> +       cr1 |= BIT_RF_OR_INTEN;
>         cr1 |= BIT_FB1_DMA_DONE_INTEN;
>         cr1 |= BIT_FB2_DMA_DONE_INTEN;
>
> @@ -262,7 +264,8 @@ static void imx7_csi_hw_disable_irq(struct imx7_csi *csi)
>  {
>         u32 cr1 = imx7_csi_reg_read(csi, CSI_CSICR1);
>
>
> -       cr1 &= ~BIT_RFF_OR_INT;
> +       cr1 &= ~BIT_SOF_INTEN;
> +       cr1 &= ~BIT_RF_OR_INTEN;
>         cr1 &= ~BIT_FB1_DMA_DONE_INTEN;
>         cr1 &= ~BIT_FB2_DMA_DONE_INTEN;
>
> In the working kernel, the csi configure enables both bits for
> BIT_BASEADDR_SWITCH_EN  and  BIT_BASEADDR_SWITCH_SEL.  I don't know
> why we want or don't want these, but these do apear to be selected in
> the custom kernel.
>
> @@ -464,7 +467,7 @@ static void imx7_csi_configure(struct imx7_csi *csi)
>         } else {
>                 cr1 |= BIT_SOF_POL | BIT_REDGE | BIT_HSYNC_POL | BIT_FCC;
>
> -               cr18 |= BIT_DATA_FROM_MIPI;
> +               cr18 |= BIT_DATA_FROM_MIPI | BIT_BASEADDR_SWITCH_EN |
> BIT_BASEADDR_SWITCH_SEL;
>
>                 switch (csi->format_mbus[IMX7_CSI_PAD_SINK].code) {
>
> In the CSIS driver, the interrupt enable in the working driver enables
> the interrupt with 0xf00fffff, but upstream is ffffffff.  Looking at
> the data sheet, both appear to me to be setting reserved values.
> The upstream driver also enables MIPI_CSIS_DBG_INTR_MSK which doesn't
> appear to be documented in the 8MM.
>
>  static void mipi_csis_enable_interrupts(struct csi_state *state, bool on)
>  {
> -       mipi_csis_write(state, MIPI_CSIS_INT_MSK, on ? 0xffffffff : 0);
> -       mipi_csis_write(state, MIPI_CSIS_DBG_INTR_MSK, on ? 0xffffffff : 0);
> +       mipi_csis_write(state, MIPI_CSIS_INT_MSK, on ? 0xf00fffff : 0);
> +       /* mipi_csis_write(state, MIPI_CSIS_DBG_INTR_MSK, on ?
> 0xffffffff : 0); */
>  }
>
> Since I am not an expert in these drivers, I am hoping someone who
> understands this better might be able to better explain what some of
> these registers do.
>
> I'll keep working again when I have more spare time.

+Lucas Stach
I think I found the issue of the camera hanging. It appears to be
related to the disp-blk-ctrl virtual power domains.

In the NXP kernel, the dispmix power domain in ATF sets bits 16 and 17
of 32e2_8008 (GPR_MIPI_RESET_DIV), but the disp-blk-ctl driver only
configures 32e2_8000 (SFT_RSTN_CSR) and 32e2_8004 (CLK_EN_CSR).  As
soon as I set those extra bits, the hanging went away, and I got an
image and the image looked good.  I'll try to work on this tomorrow or
the weekend to submit a patch with a fixes tag to make sure the
IMX8MM_DISPBLK_PD_MIPI_CSI power domain properly configures the extra
register.

With this change, I didn't need to modify either the CSI-bridge driver
nor the mipi_csi driver, so when I get the blk-ctrl fix, I'll resubmit
this series on top of that with the corrections suggested as a proper
patch without the RFC.

adam
>
> adam
>
> > > Best regards,
> > >
> > > Tim

Re: [RFC V2 0/5] arm64: dts: imx8mm: Enable CSI and OV5640 Camera

[Fabio Estevam]

Hi Adam,

On Fri, Nov 5, 2021 at 1:14 AM Adam Ford <aford173@gmail.com> wrote:

> +Lucas Stach
> I think I found the issue of the camera hanging. It appears to be
> related to the disp-blk-ctrl virtual power domains.
>
> In the NXP kernel, the dispmix power domain in ATF sets bits 16 and 17
> of 32e2_8008 (GPR_MIPI_RESET_DIV), but the disp-blk-ctl driver only
> configures 32e2_8000 (SFT_RSTN_CSR) and 32e2_8004 (CLK_EN_CSR).  As
> soon as I set those extra bits, the hanging went away, and I got an
> image and the image looked good.  I'll try to work on this tomorrow or
> the weekend to submit a patch with a fixes tag to make sure the
> IMX8MM_DISPBLK_PD_MIPI_CSI power domain properly configures the extra
> register.
>
> With this change, I didn't need to modify either the CSI-bridge driver
> nor the mipi_csi driver, so when I get the blk-ctrl fix, I'll resubmit
> this series on top of that with the corrections suggested as a proper
> patch without the RFC.

That's great news! Thanks a lot for your good work.

Cheers

Re: [RFC V2 5/5] arm64: dts: imx8mm-evk: Enable OV5640 Camera

[Tim Harvey]

On Wed, Oct 27, 2021 at 6:34 PM Laurent Pinchart
<laurent.pinchart@ideasonboard.com> wrote:
>
> Hi Adam,
>
> Thank you for the patch.
>
> On Sat, Oct 23, 2021 at 03:34:56PM -0500, Adam Ford wrote:
> > The schematic shows support for a camera interface, and the NXP
> > kernel shows it is an OV5640.
>
> The camera is an external module though. Should this be a DT overlay ?
>

Laurent,

I wanted to ask you about your comment here. I would agree that for
something like the OV5640 on the imx8mm-evk which is an add-in board
via a connector should be a dt overlay. I'm investigating using
overlays for features like this on my boards vs creating hierarchical
dts files and I see that the kernel allows building fragments with
'/plugin/' but I don't see any such overlays in the kernel tree
currently. Would overlay/fragments be accepted? Are there any examples
in the kernel tree already that I'm missing?

Best regards,

Tim

Re: [RFC V2 5/5] arm64: dts: imx8mm-evk: Enable OV5640 Camera

[Laurent Pinchart]

Hi Tim,

On Fri, Nov 05, 2021 at 09:00:03AM -0700, Tim Harvey wrote:
> On Wed, Oct 27, 2021 at 6:34 PM Laurent Pinchart wrote:
> >
> > Hi Adam,
> >
> > Thank you for the patch.
> >
> > On Sat, Oct 23, 2021 at 03:34:56PM -0500, Adam Ford wrote:
> > > The schematic shows support for a camera interface, and the NXP
> > > kernel shows it is an OV5640.
> >
> > The camera is an external module though. Should this be a DT overlay ?
> >
> 
> Laurent,
> 
> I wanted to ask you about your comment here. I would agree that for
> something like the OV5640 on the imx8mm-evk which is an add-in board
> via a connector should be a dt overlay. I'm investigating using
> overlays for features like this on my boards vs creating hierarchical
> dts files and I see that the kernel allows building fragments with
> '/plugin/' but I don't see any such overlays in the kernel tree
> currently. Would overlay/fragments be accepted?

I believe so (otherwise I'm not sure why the build system would allow
compiling overlays). Rob may have a more authoritative opinion on this
topic.

> Are there any examples
> in the kernel tree already that I'm missing?

Commit 7a4c31ee877a ("arm64: zynqmp: Add support for Xilinx Kria SOM
board") has been added recently.

-- 
Regards,

Laurent Pinchart