[PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Fabio Estevam]

The current instructions for imx6q-sabresd do not lead to functional
capture on OV5640 MIPI CSI-2.

The reason for this, as explained by Steve Longerbeam, is that OV5640 by
default transmits on virtual channel 0, not channel 1 as is given in the
instructions.

Adapt the instructions to use virtual channel 0 so that a working
camera setup can be achieved on imx6q-sabresd.

Suggested-by: Steve Longerbeam <slongerbeam@gmail.com>
Signed-off-by: Fabio Estevam <festevam@gmail.com>
---
Hi Steve,

I don't have access to a imx6q sabrelite with camera, but I think
it needs fixing as well.

 Documentation/media/v4l-drivers/imx.rst | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
index 1246573c1019..7c3afd32e456 100644
--- a/Documentation/media/v4l-drivers/imx.rst
+++ b/Documentation/media/v4l-drivers/imx.rst
@@ -645,25 +645,26 @@ The OV5640 module connects to MIPI connector J5 (sorry I don't have the
 compatible module part number or URL).
 
 The following example configures a direct conversion pipeline to capture
-from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
-can be any format supported by the OV5640. $sensordim is the frame
-dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can
-be any format supported by the ipu1_ic_prpenc entity at its output pad:
+from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
 
 .. code-block:: none
 
    # Setup links
    media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
-   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
-   media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
    media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
    media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
    # Configure pads
-   media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
-   media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
-   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prpenc':0 [fmt:AYUV32/640x480]"
+   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"
 
 Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
-- 
2.17.1

[PATCH 2/4] media: imx.rst: Provide an example for unprocessed video capture

[Fabio Estevam]

The current example for imx6q-sabresd is for a direct conversion pipeline.

Provide an extra example using unprocessed video capture for completeness.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
---
 Documentation/media/v4l-drivers/imx.rst | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
index 7c3afd32e456..3b4767f5f422 100644
--- a/Documentation/media/v4l-drivers/imx.rst
+++ b/Documentation/media/v4l-drivers/imx.rst
@@ -670,6 +670,26 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+The following example configures unprocessed video capture pipeline to
+capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
+
+.. code-block:: none
+
+   # Setup links
+   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
+   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
+   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
+   media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
+   # Configure pads
+   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
+   media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
+
+Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
+tool can be used to select any supported YUV or RGB pixelformat on the
+capture device node.
+
 
 Known Issues
 ------------
-- 
2.17.1

[PATCH 3/4] media: imx.rst: Provide the OV560 module part number

[Fabio Estevam]

In order to improve the documentation, provide the OV5640 MIPI module
part number that is used on the imx6q-sabresd board.

Signed-off-by: Fabio Estevam <festevam@gmail.com>
---
 Documentation/media/v4l-drivers/imx.rst | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
index 3b4767f5f422..37b944f12d93 100644
--- a/Documentation/media/v4l-drivers/imx.rst
+++ b/Documentation/media/v4l-drivers/imx.rst
@@ -641,8 +641,8 @@ connects to i2c bus 1 and the OV5640 to i2c bus 2.
 The device tree for SabreSD includes OF graphs for both the parallel
 OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
 CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
-The OV5640 module connects to MIPI connector J5 (sorry I don't have the
-compatible module part number or URL).
+The OV5640 module connects to MIPI connector J5. The NXP part number
+for the OV5640 module that connects to the SabreSD board is H120729.
 
 The following example configures a direct conversion pipeline to capture
 from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
-- 
2.17.1

[PATCH 4/4] media: imx.rst: Provide streaming examples for imx6q-sabresd

[Fabio Estevam]

Improve the documentation by providing examples on how to test camera
capture on imx6q-sabresd via v4l2-ctl and Gstreamer. 

Signed-off-by: Fabio Estevam <festevam@gmail.com>
---
 Documentation/media/v4l-drivers/imx.rst | 42 +++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
index 37b944f12d93..c872cc84d9fc 100644
--- a/Documentation/media/v4l-drivers/imx.rst
+++ b/Documentation/media/v4l-drivers/imx.rst
@@ -670,6 +670,28 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+In order to determine what is the /dev/video node correspondent to
+"ipu1_ic_prpenc":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_ic_prpenc | grep Registered
+   ipu1_ic_prpenc: Registered ipu1_ic_prpenc capture as /dev/video1
+
+/dev/video1 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video1
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
+
 The following example configures unprocessed video capture pipeline to
 capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
 
@@ -690,6 +712,26 @@ Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
 tool can be used to select any supported YUV or RGB pixelformat on the
 capture device node.
 
+In order to determine what is the /dev/video node correspondent to "ipu1_csi0":
+
+.. code-block:: none
+
+   # dmesg | grep ipu1_csi0 | grep Registered
+   ipu1_csi0: Registered ipu1_csi0 capture as /dev/video0
+
+/dev/video0 is the streaming element in this case.
+
+Starting the streaming via v4l2-ctl:
+
+.. code-block:: none
+
+   v4l2-ctl --stream-mmap -d /dev/video0
+
+Starting the streaming via Gstreamer and sending the content to the display:
+
+.. code-block:: none
+
+   gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
 
 Known Issues
 ------------
-- 
2.17.1

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Steve Longerbeam]

Hi Fabio, thanks for the patch.

On 4/8/20 5:32 AM, Fabio Estevam wrote:
> The current instructions for imx6q-sabresd do not lead to functional
> capture on OV5640 MIPI CSI-2.
>
> The reason for this, as explained by Steve Longerbeam, is that OV5640 by
> default transmits on virtual channel 0, not channel 1 as is given in the
> instructions.
>
> Adapt the instructions to use virtual channel 0 so that a working
> camera setup can be achieved on imx6q-sabresd.
>
> Suggested-by: Steve Longerbeam <slongerbeam@gmail.com>
> Signed-off-by: Fabio Estevam <festevam@gmail.com>
> ---
> Hi Steve,
>
> I don't have access to a imx6q sabrelite with camera, but I think
> it needs fixing as well.
>
>   Documentation/media/v4l-drivers/imx.rst | 23 ++++++++++++-----------
>   1 file changed, 12 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
> index 1246573c1019..7c3afd32e456 100644
> --- a/Documentation/media/v4l-drivers/imx.rst
> +++ b/Documentation/media/v4l-drivers/imx.rst
> @@ -645,25 +645,26 @@ The OV5640 module connects to MIPI connector J5 (sorry I don't have the
>   compatible module part number or URL).
>   
>   The following example configures a direct conversion pipeline to capture
> -from the OV5640, transmitting on MIPI CSI-2 virtual channel 1. $sensorfmt
> -can be any format supported by the OV5640. $sensordim is the frame
> -dimension part of $sensorfmt (minus the mbus pixel code). $outputfmt can
> -be any format supported by the ipu1_ic_prpenc entity at its output pad:
> +from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:

Since this is an Image Converter pipeline, it would be nice to 
demonstrate scaling and color conversion in this example. How about 
changing the format at IC output to be ARGB8888_1X32/800x600. So change 
the explanation of the pipeline above, and:

>   
>   .. code-block:: none
>   
>      # Setup links
>      media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
> -   media-ctl -l "'imx6-mipi-csi2':2 -> 'ipu1_csi1':0[1]"
> -   media-ctl -l "'ipu1_csi1':1 -> 'ipu1_ic_prp':0[1]"
> +   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
> +   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
> +   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
>      media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
>      media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
>      # Configure pads
> -   media-ctl -V "'ov5640 1-003c':0 [fmt:$sensorfmt field:none]"
> -   media-ctl -V "'imx6-mipi-csi2':2 [fmt:$sensorfmt field:none]"
> -   media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/$sensordim field:none]"
> -   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/$sensordim field:none]"
> -   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:$outputfmt field:none]"
> +   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'ipu1_csi0':0 [fmt:UYVY2X8/640x480]"

don't need this line (I pulled this from a script that was testing 
crop/compose on sink pad ipu1_csi0:0, but since we're not doing that 
here, you can remove the format setup line for ipu1_csi0:0).

> +   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
> +   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
> +   media-ctl -V "'ipu1_ic_prpenc':0 [fmt:AYUV32/640x480]"

don't need this line.

> +   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"

change to:

media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"


>   
>   Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
>   tool can be used to select any supported YUV or RGB pixelformat on the

"...The v4l2-ctl tool can be used to select any supported RGB pixelformat..."


Steve

Re: [PATCH 2/4] media: imx.rst: Provide an example for unprocessed video capture

[Steve Longerbeam]

On 4/8/20 5:32 AM, Fabio Estevam wrote:
> The current example for imx6q-sabresd is for a direct conversion pipeline.
>
> Provide an extra example using unprocessed video capture for completeness.
>
> Signed-off-by: Fabio Estevam <festevam@gmail.com>

Reviewed-by: Steve Longerbeam <slongerbeam@gmail.com>

> ---
>   Documentation/media/v4l-drivers/imx.rst | 20 ++++++++++++++++++++
>   1 file changed, 20 insertions(+)
>
> diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
> index 7c3afd32e456..3b4767f5f422 100644
> --- a/Documentation/media/v4l-drivers/imx.rst
> +++ b/Documentation/media/v4l-drivers/imx.rst
> @@ -670,6 +670,26 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
>   tool can be used to select any supported YUV or RGB pixelformat on the
>   capture device node.
>   
> +The following example configures unprocessed video capture pipeline to
> +capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
> +
> +.. code-block:: none
> +
> +   # Setup links
> +   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
> +   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
> +   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
> +   media-ctl -l "'ipu1_csi0':2 -> 'ipu1_csi0 capture':0[1]"
> +   # Configure pads
> +   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'ipu1_csi0_mux':0 [fmt:UYVY2X8/640x480]"
> +   media-ctl -V "'ipu1_csi0':0 [fmt:AYUV32/640x480]"
> +
> +Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
> +tool can be used to select any supported YUV or RGB pixelformat on the
> +capture device node.
> +
>   
>   Known Issues
>   ------------

Re: [PATCH 3/4] media: imx.rst: Provide the OV560 module part number

[Steve Longerbeam]

On 4/8/20 5:32 AM, Fabio Estevam wrote:
> In order to improve the documentation, provide the OV5640 MIPI module
> part number that is used on the imx6q-sabresd board.
>
> Signed-off-by: Fabio Estevam <festevam@gmail.com>

Reviewed-by: Steve Longerbeam <slongerbeam@gmail.com>

> ---
>   Documentation/media/v4l-drivers/imx.rst | 4 ++--
>   1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
> index 3b4767f5f422..37b944f12d93 100644
> --- a/Documentation/media/v4l-drivers/imx.rst
> +++ b/Documentation/media/v4l-drivers/imx.rst
> @@ -641,8 +641,8 @@ connects to i2c bus 1 and the OV5640 to i2c bus 2.
>   The device tree for SabreSD includes OF graphs for both the parallel
>   OV5642 and the MIPI CSI-2 OV5640, but as of this writing only the MIPI
>   CSI-2 OV5640 has been tested, so the OV5642 node is currently disabled.
> -The OV5640 module connects to MIPI connector J5 (sorry I don't have the
> -compatible module part number or URL).
> +The OV5640 module connects to MIPI connector J5. The NXP part number
> +for the OV5640 module that connects to the SabreSD board is H120729.
>   
>   The following example configures a direct conversion pipeline to capture
>   from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:

Re: [PATCH 4/4] media: imx.rst: Provide streaming examples for imx6q-sabresd

[Steve Longerbeam]

On 4/8/20 5:32 AM, Fabio Estevam wrote:
> Improve the documentation by providing examples on how to test camera
> capture on imx6q-sabresd via v4l2-ctl and Gstreamer.
>
> Signed-off-by: Fabio Estevam <festevam@gmail.com>
> ---
>   Documentation/media/v4l-drivers/imx.rst | 42 +++++++++++++++++++++++++
>   1 file changed, 42 insertions(+)
>
> diff --git a/Documentation/media/v4l-drivers/imx.rst b/Documentation/media/v4l-drivers/imx.rst
> index 37b944f12d93..c872cc84d9fc 100644
> --- a/Documentation/media/v4l-drivers/imx.rst
> +++ b/Documentation/media/v4l-drivers/imx.rst
> @@ -670,6 +670,28 @@ Streaming can then begin on "ipu1_ic_prpenc capture" node. The v4l2-ctl
>   tool can be used to select any supported YUV or RGB pixelformat on the
>   capture device node.
>   
> +In order to determine what is the /dev/video node correspondent to
> +"ipu1_ic_prpenc":
> +
> +.. code-block:: none
> +
> +   # dmesg | grep ipu1_ic_prpenc | grep Registered

A better way is to use "media-ctl -e":

# media-ctl -e "ipu1_ic_prpenc capture"
/dev/video1

Grepping from dmesg can fail when the log fills up and pushes out the 
boot "Registered ..." messages.

Steve

> +   ipu1_ic_prpenc: Registered ipu1_ic_prpenc capture as /dev/video1
> +
> +/dev/video1 is the streaming element in this case.
> +
> +Starting the streaming via v4l2-ctl:
> +
> +.. code-block:: none
> +
> +   v4l2-ctl --stream-mmap -d /dev/video1
> +
> +Starting the streaming via Gstreamer and sending the content to the display:
> +
> +.. code-block:: none
> +
> +   gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink
> +
>   The following example configures unprocessed video capture pipeline to
>   capture from the OV5640, transmitting on MIPI CSI-2 virtual channel 0:
>   
> @@ -690,6 +712,26 @@ Streaming can then begin on "ipu1_csi0 capture" node. The v4l2-ctl
>   tool can be used to select any supported YUV or RGB pixelformat on the
>   capture device node.
>   
> +In order to determine what is the /dev/video node correspondent to "ipu1_csi0":
> +
> +.. code-block:: none
> +
> +   # dmesg | grep ipu1_csi0 | grep Registered
> +   ipu1_csi0: Registered ipu1_csi0 capture as /dev/video0
> +
> +/dev/video0 is the streaming element in this case.
> +
> +Starting the streaming via v4l2-ctl:
> +
> +.. code-block:: none
> +
> +   v4l2-ctl --stream-mmap -d /dev/video0
> +
> +Starting the streaming via Gstreamer and sending the content to the display:
> +
> +.. code-block:: none
> +
> +   gst-launch-1.0 v4l2src device=/dev/video0 ! kmssink
>   
>   Known Issues
>   ------------

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Fabio Estevam]

Hi Steve,

On Wed, Apr 8, 2020 at 1:35 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:

> change to:
>
> media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

If I make this change I get:

# v4l2-ctl --stream-mmap -d /dev/video1
[  501.143012] ipu1_ic_prpenc: capture format not valid
                VIDIOC_STREAMON returned -1 (Invalid argument)

Going back to the original example:

#  media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"
# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

Changing only the resolution:

#  media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/800x600]"
# v4l2-ctl --stream-mmap -d /dev/video1
[  658.358246] ipu1_ic_prpenc: capture format not valid
                VIDIOC_STREAMON returned -1 (Invalid argument)

Any ideas?

These are the lines I am currently using as per your feedback:

   # Setup links
   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
   media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
   media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
   # Configure pads
   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Thanks,

Fabio Estevam

   # Setup links
   media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
   media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
   media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
   media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
   media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
   media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
   # Configure pads
   media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
   media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
   media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
   media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Steve Longerbeam]

On 4/8/20 10:24 AM, Fabio Estevam wrote:
> Hi Steve,
>
> On Wed, Apr 8, 2020 at 1:35 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:
>
>> change to:
>>
>> media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
> If I make this change I get:
>
> # v4l2-ctl --stream-mmap -d /dev/video1
> [  501.143012] ipu1_ic_prpenc: capture format not valid
>                  VIDIOC_STREAMON returned -1 (Invalid argument)
>
> Going back to the original example:
>
> #  media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/640x480]"
> # v4l2-ctl --stream-mmap -d /dev/video1
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
>
> Changing only the resolution:
>
> #  media-ctl -V "'ipu1_ic_prpenc':1 [fmt:AYUV32/800x600]"
> # v4l2-ctl --stream-mmap -d /dev/video1
> [  658.358246] ipu1_ic_prpenc: capture format not valid
>                  VIDIOC_STREAMON returned -1 (Invalid argument)
>
> Any ideas?

You have to set a format at the capture interface as the last step 
before streaming can start:

# v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3


or whatever RGB format is supported. It's enough to sinmply set one parameter, such as width or height or pixelformat. The driver will propagate the reset from the source subdev ipu1_ic_prpenc.

The capture interface used to be automatically propagated from the source subdev, but now it must be done by userland.

Steve

>
> These are the lines I am currently using as per your feedback:
>
>     # Setup links
>     media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
>     media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
>     media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
>     media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
>     media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
>     media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
>     # Configure pads
>     media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
>     media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
>     media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"
>
> Thanks,
>
> Fabio Estevam
>
>     # Setup links
>     media-ctl -l "'ov5640 1-003c':0 -> 'imx6-mipi-csi2':0[1]"
>     media-ctl -l "'imx6-mipi-csi2':1 -> 'ipu1_csi0_mux':0[1]"
>     media-ctl -l "'ipu1_csi0_mux':2 -> 'ipu1_csi0':0[1]"
>     media-ctl -l "'ipu1_csi0':1 -> 'ipu1_ic_prp':0[1]"
>     media-ctl -l "'ipu1_ic_prp':1 -> 'ipu1_ic_prpenc':0[1]"
>     media-ctl -l "'ipu1_ic_prpenc':1 -> 'ipu1_ic_prpenc capture':0[1]"
>     # Configure pads
>     media-ctl -V "'ov5640 1-003c':0 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'imx6-mipi-csi2':1 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'ipu1_csi0_mux':2 [fmt:UYVY2X8/640x480]"
>     media-ctl -V "'ipu1_csi0':1 [fmt:AYUV32/640x480]"
>     media-ctl -V "'ipu1_ic_prp':1 [fmt:AYUV32/640x480]"
>     media-ctl -V "'ipu1_ic_prpenc':1 [fmt:ARGB8888_1X32/800x600]"

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Fabio Estevam]

Hi Steve,

On Wed, Apr 8, 2020 at 4:16 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:

> You have to set a format at the capture interface as the last step
> before streaming can start:
>
> # v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3

Thanks. After running this line the stream can start via --stream-mmap:

# v4l2-ctl --stream-mmap -d /dev/video1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps

but it fails with Gstreamer:

# gst-launch-1.0 -v v4l2src device=/dev/video1 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1024
/GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 768
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn't want to preroll.
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Should I change the Gstreamer pipeline?

Thanks

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Fabio Estevam]

On Wed, Apr 8, 2020 at 4:28 PM Fabio Estevam <festevam@gmail.com> wrote:

> Should I change the Gstreamer pipeline?

It works if I use the v4l2convert element:

gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink

I will send it in the next patch series.

Thanks

>
> Thanks

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Steve Longerbeam]

On 4/8/20 12:28 PM, Fabio Estevam wrote:
> Hi Steve,
>
> On Wed, Apr 8, 2020 at 4:16 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:
>
>> You have to set a format at the capture interface as the last step
>> before streaming can start:
>>
>> # v4l2-ctl -d /dev/video1 --set-fmt-video=pixelformat=RGB3
> Thanks. After running this line the stream can start via --stream-mmap:
>
> # v4l2-ctl --stream-mmap -d /dev/video1
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
>
> but it fails with Gstreamer:
>
> # gst-launch-1.0 -v v4l2src device=/dev/video1 ! kmssink
> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 1024
> /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 768
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Internal data stream error.
> Additional debug info:
> ../libs/gst/base/gstbasesrc.c(3072): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> streaming stopped, reason not-negotiated (-4)
> ERROR: pipeline doesn't want to preroll.
> Setting pipeline to PAUSED ...
> Setting pipeline to READY ...
> Setting pipeline to NULL ...
> Freeing pipeline ...
>
> Should I change the Gstreamer pipeline?

It works for me, I'm running gstreamer version:

arm:~/scripts % gst-launch-1.0 --version
gst-launch-1.0 version 1.14.5
GStreamer 1.14.5
https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

Also, I'm streaming to an HDMI monitor at 1920x1080 (passing 
connector-id=54 to kmssink).

I can't tell from your gst output whether you are having an issue with 
pixelformats or frame dimensions, but anyway here is my gst pipeline:

gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 
name=imx-drm sync=0 can-scale=false

It might also be the can-scale property, you might need to provide 
can-scale=false to tell kmssink that imx-drm cannot scale 800x600 to 
something else.

While streaming, I checked what pixelformat the pipeline finally settled on:

arm:~/scripts % v4l2-ctl -d1 -V
Format Video Capture:
     Width/Height      : 800/600
     Pixel Format      : 'XR24'
     Field             : None
     Bytes per Line    : 3200
     Size Image        : 1920000
     Colorspace        : sRGB
     Transfer Function : sRGB
     YCbCr/HSV Encoding: ITU-R 601
     Quantization      : Full Range
     Flags             :

Steve

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Steve Longerbeam]

On 4/8/20 12:33 PM, Fabio Estevam wrote:
> On Wed, Apr 8, 2020 at 4:28 PM Fabio Estevam <festevam@gmail.com> wrote:
>
>> Should I change the Gstreamer pipeline?
> It works if I use the v4l2convert element:
>
> gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink

I think that works because the csc-scaler mem2mem driver is able to 
provide the frame dimensions that kmssink requires, but I don't think 
you need v4l2convert, I think this should work (use can-scale=false):

gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 
name=imx-drm sync=0 can-scale=false

but use the connector-id for your display.

Steve

Re: [PATCH 1/4] media: imx.rst: Fix the MIPI CSI-2 virtual channel

[Steve Longerbeam]

Hi Fabio,

So sorry!

I was testing with the patchset:

[PATCH v7 00/11] media: imx: Miscellaneous format-related cleanups

which specifically fixes a bug with enumerating non-mbus RGB formats:

[PATCH v7 01/11] media: imx: utils: fix and simplify pixel format 
enumeration

Without that patchset, /dev/video1 only lists:

arm:~/scripts % v4l2-ctl -d1 --list-formats
ioctl: VIDIOC_ENUM_FMT
     Index       : 0
     Type        : Video Capture
     Pixel Format: 'RGBP'
     Name        : 16-bit RGB 5-6-5

     Index       : 1
     Type        : Video Capture
     Pixel Format: 'RGB3'
     Name        : 24-bit RGB 8-8-8

     Index       : 2
     Type        : Video Capture
     Pixel Format: 'BX24'
     Name        : 32-bit XRGB 8-8-8-8

But with the bug fix:

arm:~ % v4l2-ctl -d1 --list-formats
ioctl: VIDIOC_ENUM_FMT
         Index       : 0
         Type        : Video Capture
         Pixel Format: 'RGBP'
         Name        : 16-bit RGB 5-6-5

         Index       : 1
         Type        : Video Capture
         Pixel Format: 'RGB3'
         Name        : 24-bit RGB 8-8-8

         Index       : 2
         Type        : Video Capture
         Pixel Format: 'BGR3'
         Name        : 24-bit BGR 8-8-8

         Index       : 3
         Type        : Video Capture
         Pixel Format: 'BX24'
         Name        : 32-bit XRGB 8-8-8-8

         Index       : 4
         Type        : Video Capture
         Pixel Format: 'XR24'
         Name        : 32-bit BGRX 8-8-8-8

         Index       : 5
         Type        : Video Capture
         Pixel Format: 'RX24'
         Name        : 32-bit XBGR 8-8-8-8

         Index       : 6
         Type        : Video Capture
         Pixel Format: 'XB24'
         Name        : 32-bit RGBX 8-8-8-8


And with that, kmssink is able to negotiate XR24.

With the anticipation that the patchset mentioned will be merged, can 
you try again with that patchset applied, and update the imx.rst doc 
accordingly?

Thanks,
Steve


On 4/8/20 12:51 PM, Steve Longerbeam wrote:
>
>
> On 4/8/20 12:33 PM, Fabio Estevam wrote:
>> On Wed, Apr 8, 2020 at 4:28 PM Fabio Estevam <festevam@gmail.com> wrote:
>>
>>> Should I change the Gstreamer pipeline?
>> It works if I use the v4l2convert element:
>>
>> gst-launch-1.0 -v v4l2src device=/dev/video1 ! v4l2convert ! kmssink
>
> I think that works because the csc-scaler mem2mem driver is able to 
> provide the frame dimensions that kmssink requires, but I don't think 
> you need v4l2convert, I think this should work (use can-scale=false):
>
> gst-launch-1.0 v4l2src device=/dev/video1 ! kmssink connector-id=54 
> name=imx-drm sync=0 can-scale=false
>
> but use the connector-id for your display.
>
> Steve
>