imx6 capture via ADV7280

imx6 capture via ADV7280

[Fabio Estevam]

Hi,

I am running 5.10.0-rc5-next-20201125 (same behavior is seen on
5.4.76) on a imx6dl board with an ADV7280:

# media-ctl -l "'adv7180 0-0020':0 -> 'ipu1_csi1_mux':4[1]";
# media-ctl -l "'ipu1_csi1_mux':5 -> 'ipu1_csi1':0[1]"
# media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
# media-ctl -V "'adv7180 0-0020':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
# media-ctl -V "'ipu1_csi1_mux':5 [fmt:UYVY2X8/720x480]"
# media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/720x480]"
# v4l2-ctl -d3 --set-fmt-video=field=interlaced_bt

Then I start the stream:
v4l2-ctl --stream-mmap -d /dev/video3
[   22.792049] ipu1_csi1: EOF timeout
VIDIOC_DQBUF: failed: Input/output error

It always fails for the first time.

Then I try to start the stream once again:

# v4l2-ctl --stream-mmap -d /dev/video3
[   28.430420] ipu1_csi1: NFB4EOF
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.75 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.78 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.78 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
....

Does anyone have an idea as to why it fails when trying to stream for
the first time?

Next, I would like to run the following Gstreamer pipeline (GStreamer 1.18.1):

# gst-launch-1.0 v4l2src device=/dev/video3 ! kmssink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device
'/dev/video3' does not support progressive interlacing
Additional debug info:
../sys/v4l2/gstv4l2object.c(4012): gst_v4l2_object_set_format_full ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Device wants interleaved interlacing
Execution ended after 0:00:00.014920000
Setting pipeline to NULL ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Freeing pipeline ...

Any ideas on how to display the video captured to the display?

Thanks,

Fabio Estevam

Re: imx6 capture via ADV7280

[Nicolas Dufresne]

Le mercredi 25 novembre 2020 à 17:47 -0300, Fabio Estevam a écrit :
> Hi,
> 
> I am running 5.10.0-rc5-next-20201125 (same behavior is seen on
> 5.4.76) on a imx6dl board with an ADV7280:
> 
> # media-ctl -l "'adv7180 0-0020':0 -> 'ipu1_csi1_mux':4[1]";
> # media-ctl -l "'ipu1_csi1_mux':5 -> 'ipu1_csi1':0[1]"
> # media-ctl -l "'ipu1_csi1':2 -> 'ipu1_csi1 capture':0[1]"
> # media-ctl -V "'adv7180 0-0020':0 [fmt:UYVY2X8/720x480 field:seq-bt]"
> # media-ctl -V "'ipu1_csi1_mux':5 [fmt:UYVY2X8/720x480]"
> # media-ctl -V "'ipu1_csi1':2 [fmt:AYUV32/720x480]"
> # v4l2-ctl -d3 --set-fmt-video=field=interlaced_bt
> 
> Then I start the stream:
> v4l2-ctl --stream-mmap -d /dev/video3
> [   22.792049] ipu1_csi1: EOF timeout
> VIDIOC_DQBUF: failed: Input/output error
> 
> It always fails for the first time.
> 
> Then I try to start the stream once again:
> 
> # v4l2-ctl --stream-mmap -d /dev/video3
> [   28.430420] ipu1_csi1: NFB4EOF
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.75 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.78 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.78 fps
> <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.77 fps
> ....
> 
> Does anyone have an idea as to why it fails when trying to stream for
> the first time?
> 
> Next, I would like to run the following Gstreamer pipeline (GStreamer 1.18.1):
> 
> # gst-launch-1.0 v4l2src device=/dev/video3 ! kmssink

I support kmssink ask for progressive frames, while v4l2src can only produce
interlaced. There is likely something to improve there, but meanwhile, consider
usign deinterlace element.

> Setting pipeline to PAUSED ...
> Pipeline is live and does not need PREROLL ...
> Pipeline is PREROLLED ...
> Setting pipeline to PLAYING ...
> New clock: GstSystemClock
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device
> '/dev/video3' does not support progressive interlacing
> Additional debug info:
> ../sys/v4l2/gstv4l2object.c(4012): gst_v4l2_object_set_format_full ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Device wants interleaved interlacing
> Execution ended after 0:00:00.014920000
> Setting pipeline to NULL ...
> ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> Internal data stream error.
> Additional debug info:
> ../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop ():
> /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
> streaming stopped, reason not-negotiated (-4)
> Freeing pipeline ...
> 
> Any ideas on how to display the video captured to the display?
> 
> Thanks,
> 
> Fabio Estevam

Re: imx6 capture via ADV7280

[Fabio Estevam]

Hi Nicolas,

On Thu, Nov 26, 2020 at 3:25 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:

> I support kmssink ask for progressive frames, while v4l2src can only produce
> interlaced. There is likely something to improve there, but meanwhile, consider
> usign deinterlace element.

Thanks for your feedback. I am configuring the links like this now:

media-ctl -l "'adv7180 0-0020':0 -> 'ipu1_csi1_mux':4[1]"
media-ctl -l "'ipu1_csi1_mux':5 -> 'ipu1_csi1':0[1]"
media-ctl -l "'ipu1_csi1':1 -> 'ipu1_vdic':0[1]"
media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
media-ctl -V "'adv7180 0-0020':0 [fmt:UYVY2X8/640x480 field:seq-tb]"
media-ctl -V "'ipu1_csi1_mux':5 [fmt:UYVY2X8/640x480]"
media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/640x480]"
media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/640x480 field:none]"
media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/640x480 field:none]"
media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/640x480 field:none]"
v4l2-ctl -d2 --set-fmt-video=field=none

gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink:

And the Gstreamer pipeline works now.

Also, to fix the timeout issue I reported earlier, I had to increase
IMX_MEDIA_EOF_TIMEOUT:

--- a/drivers/staging/media/imx/imx-media.h
+++ b/drivers/staging/media/imx/imx-media.h
@@ -65,7 +65,7 @@ enum {
 };

 /* How long to wait for EOF interrupts in the buffer-capture subdevs */
-#define IMX_MEDIA_EOF_TIMEOUT       1000
+#define IMX_MEDIA_EOF_TIMEOUT       2000

 struct imx_media_pixfmt {
  u32     fourcc;
-- 
2.17.1

Will submit this change as a formal patch.

Thanks,

Fabio Estevam

Re: imx6 capture via ADV7280

[Tim Harvey]

On Fri, Nov 27, 2020 at 7:58 AM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi Nicolas,
>
> On Thu, Nov 26, 2020 at 3:25 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
> > I support kmssink ask for progressive frames, while v4l2src can only produce
> > interlaced. There is likely something to improve there, but meanwhile, consider
> > usign deinterlace element.
>
> Thanks for your feedback. I am configuring the links like this now:
>
> media-ctl -l "'adv7180 0-0020':0 -> 'ipu1_csi1_mux':4[1]"
> media-ctl -l "'ipu1_csi1_mux':5 -> 'ipu1_csi1':0[1]"
> media-ctl -l "'ipu1_csi1':1 -> 'ipu1_vdic':0[1]"
> media-ctl -l "'ipu1_vdic':2 -> 'ipu1_ic_prp':0[1]"
> media-ctl -l "'ipu1_ic_prp':2 -> 'ipu1_ic_prpvf':0[1]"
> media-ctl -l "'ipu1_ic_prpvf':1 -> 'ipu1_ic_prpvf capture':0[1]"
> media-ctl -V "'adv7180 0-0020':0 [fmt:UYVY2X8/640x480 field:seq-tb]"
> media-ctl -V "'ipu1_csi1_mux':5 [fmt:UYVY2X8/640x480]"
> media-ctl -V "'ipu1_csi1':1 [fmt:AYUV32/640x480]"
> media-ctl -V "'ipu1_vdic':2 [fmt:AYUV32/640x480 field:none]"
> media-ctl -V "'ipu1_ic_prp':2 [fmt:AYUV32/640x480 field:none]"
> media-ctl -V "'ipu1_ic_prpvf':1 [fmt:AYUV32/640x480 field:none]"
> v4l2-ctl -d2 --set-fmt-video=field=none
>
> gst-launch-1.0 v4l2src device=/dev/video2 ! kmssink:
>
> And the Gstreamer pipeline works now.
>

Fabio,

I wanted to take a look at this as you are using an adv7280 with the
adv7180 driver and I previously had found using the adv7280 on an
imx6q/dl there was a difference in the output format for the adv7280
that caused an issue with the imx6 format [1].

I have also still have to patch imx-media-csi.c to skip the first few
frames in order to sync properly on bt656 sources [2].

Did you have any patches in the imx-media or adv7180 drivers to deal
with these issues I run into?

Best Regards,

Tim
[1] - https://patchwork.kernel.org/project/linux-media/patch/20190827215539.1286-1-mmichilot@gateworks.com
[2] - https://github.com/Gateworks/linux-imx6/commit/959fbd4