imx6: Cannot register mem2mem

imx6: Cannot register mem2mem

[Fabio Estevam]

Hi,

I am running kernel 5.6.2 on an imx6qp sabresd, but I cannot get the
mem2mem element from imx-media-csc-scaler.c to probe.

I don't see imx6_media_probe_complete() getting called, hence
imx_media_csc_scaler_device_init() is never called and no mem2mem
element is registered.

Any ideas as to how to get mem2mem registered on i.MX6?

Thanks,

Fabio Estevam

Re: imx6: Cannot register mem2mem

[Fabio Estevam]

On Mon, Apr 6, 2020 at 10:37 AM Fabio Estevam <festevam@gmail.com> wrote:
>
> Hi,
>
> I am running kernel 5.6.2 on an imx6qp sabresd, but I cannot get the
> mem2mem element from imx-media-csc-scaler.c to probe.
>
> I don't see imx6_media_probe_complete() getting called, hence
> imx_media_csc_scaler_device_init() is never called and no mem2mem
> element is registered.
>
> Any ideas as to how to get mem2mem registered on i.MX6?

It seems that v4l2_async_notifier_can_complete() is always false, so
v4l2_async_notifier_call_complete() is never called.

If I change it like this:

--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -213,10 +213,6 @@ v4l2_async_notifier_try_complete(struct
v4l2_async_notifier *notifier)
        if (!notifier->v4l2_dev)
                return 0;

-       /* Is everything ready? */
-       if (!v4l2_async_notifier_can_complete(notifier))
-               return 0;
-
        return v4l2_async_notifier_call_complete(notifier);
 }

Then the mem2mem driver can be successfully probed:

[    4.601350] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video11

Any suggestions?

Thanks

Re: imx6: Cannot register mem2mem

[Steve Longerbeam]

Hi Fabio,

On 4/6/20 6:37 AM, Fabio Estevam wrote:
> Hi,
>
> I am running kernel 5.6.2 on an imx6qp sabresd, but I cannot get the
> mem2mem element from imx-media-csc-scaler.c to probe.
>
> I don't see imx6_media_probe_complete() getting called, hence
> imx_media_csc_scaler_device_init() is never called and no mem2mem
> element is registered.
>
> Any ideas as to how to get mem2mem registered on i.MX6?

If you're sabresd does not have the OV5640 module attached, then probe 
won't complete.

You could make the upstream remote availability optional in the 
imx6-mipi-csi2 receiver driver:

--- a/drivers/staging/media/imx/imx6-mipi-csi2.c
+++ b/drivers/staging/media/imx/imx6-mipi-csi2.c
@@ -537,10 +537,8 @@ static int csi2_parse_endpoint(struct device *dev,
         struct v4l2_subdev *sd = dev_get_drvdata(dev);
         struct csi2_dev *csi2 = sd_to_dev(sd);

-       if (!fwnode_device_is_available(asd->match.fwnode)) {
-               v4l2_err(&csi2->sd, "remote is not available\n");
-               return -EINVAL;
-       }
+       if (!fwnode_device_is_available(asd->match.fwnode))
+               return -ENOTCONN;

         if (vep->bus_type != V4L2_MBUS_CSI2_DPHY) {
                 v4l2_err(&csi2->sd, "invalid bus type, must be MIPI 
CSI2\n");


Another option would be to disable the mipi-csi2 receiver in the device 
tree:

--- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
+++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
@@ -158,7 +158,7 @@
  };

  &mipi_csi {
-       status = "okay";
+       status = "disabled";

         port@0 {
                 reg = <0>;



The first patch doesn't make much sense though, without a remote sensor 
the mipi-csi2 receiver won't function, so there's no point in taking up 
resources used by the driver.

The second option will require migrating the mipi-csi2 receiver 
enablement to new target .dts files, for example a new 
"imx6qp-sabresd-mipi.dts". That has already been done for some boards, 
for example the imx6qdl-icore has a separate target imx6q-icore-mipi, 
that enables mipi-csi2.


Steve

Re: imx6: Cannot register mem2mem

[Fabio Estevam]

Hi Steve,

On Mon, Apr 6, 2020 at 2:36 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:

> If you're sabresd does not have the OV5640 module attached, then probe
> won't complete.

Yes, correct.

The imx6qp sabresd I am using does not have the OV5640 cameras attached to it.

> Another option would be to disable the mipi-csi2 receiver in the device
> tree:
>
> --- a/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
> +++ b/arch/arm/boot/dts/imx6qdl-sabresd.dtsi
> @@ -158,7 +158,7 @@
>   };
>
>   &mipi_csi {
> -       status = "okay";
> +       status = "disabled";

I tried this suggestion and then mem2mem is successfully probed.

Thanks for your clarification!

Re: imx6: Cannot register mem2mem

[Fabio Estevam]

Hi Steve,

On Mon, Apr 6, 2020 at 2:46 PM Fabio Estevam <festevam@gmail.com> wrote:

> >   &mipi_csi {
> > -       status = "okay";
> > +       status = "disabled";
>
> I tried this suggestion and then mem2mem is successfully probed.

Ok, so now I connected the ov5640 module as per your suggestion and
this is what I get with a clean 5.6.2:

[    6.618296] imx-media: ov5640 1-003c:0 -> imx6-mipi-csi2:0
[    6.644522] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video8

However,  the v4l2video8convert Gstreamer element is not detected:

# gst-inspect-1.0 | grep convert
video4linux2:  v4l2convert: V4L2 Video Converter
audioconvert:  audioconvert: Audio converter

If I do the kernel hack I mentioned previously then I can get v4l2video8convert:

--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -213,10 +213,6 @@ v4l2_async_notifier_try_complete(struct
v4l2_async_notifier *notifier)
        if (!notifier->v4l2_dev)
                return 0;

-       /* Is everything ready? */
-       if (!v4l2_async_notifier_can_complete(notifier))
-               return 0;
-
        return v4l2_async_notifier_call_complete(notifier);
 }

# gst-inspect-1.0 | grep convert
video4linux2:  v4l2video11convert: V4L2 Video Converter
video4linux2:  v4l2video10convert: V4L2 Video Converter
video4linux2:  v4l2video9convert: V4L2 Video Converter
video4linux2:  v4l2convert: V4L2 Video Converter
audioconvert:  audioconvert: Audio converter

What am I missing in order to get the v4l2videoXconvert Gstreamer
element without hacking the kernel?

Thanks

Re: imx6: Cannot register mem2mem

[Steve Longerbeam]

On 4/6/20 11:39 AM, Fabio Estevam wrote:
> Hi Steve,
>
> On Mon, Apr 6, 2020 at 2:46 PM Fabio Estevam <festevam@gmail.com> wrote:
>
>>>    &mipi_csi {
>>> -       status = "okay";
>>> +       status = "disabled";
>> I tried this suggestion and then mem2mem is successfully probed.
> Ok, so now I connected the ov5640 module as per your suggestion and
> this is what I get with a clean 5.6.2:
>
> [    6.618296] imx-media: ov5640 1-003c:0 -> imx6-mipi-csi2:0
> [    6.644522] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video8

This shows that imx-media completed probe (v4l2 core called its probe 
complete callback).

>
> However,  the v4l2video8convert Gstreamer element is not detected:
>
> # gst-inspect-1.0 | grep convert
> video4linux2:  v4l2convert: V4L2 Video Converter
> audioconvert:  audioconvert: Audio converter
>
> If I do the kernel hack I mentioned previously then I can get v4l2video8convert:
>
> --- a/drivers/media/v4l2-core/v4l2-async.c
> +++ b/drivers/media/v4l2-core/v4l2-async.c
> @@ -213,10 +213,6 @@ v4l2_async_notifier_try_complete(struct
> v4l2_async_notifier *notifier)
>          if (!notifier->v4l2_dev)
>                  return 0;
>
> -       /* Is everything ready? */
> -       if (!v4l2_async_notifier_can_complete(notifier))
> -               return 0;
> -
>          return v4l2_async_notifier_call_complete(notifier);

That's very odd, because v4l2_async_notifier_call_complete() was 
definitely called on the imx-media root notifier from above.

Is there is another v4l2 driver that is not completing?

Steve

>   }
>
> # gst-inspect-1.0 | grep convert
> video4linux2:  v4l2video11convert: V4L2 Video Converter
> video4linux2:  v4l2video10convert: V4L2 Video Converter
> video4linux2:  v4l2video9convert: V4L2 Video Converter
> video4linux2:  v4l2convert: V4L2 Video Converter
> audioconvert:  audioconvert: Audio converter
>
> What am I missing in order to get the v4l2videoXconvert Gstreamer
> element without hacking the kernel?
>
> Thanks

Re: imx6: Cannot register mem2mem

[Fabio Estevam]

[Adding Nicolas]

On Mon, Apr 6, 2020 at 3:50 PM Steve Longerbeam <slongerbeam@gmail.com> wrote:

> > Ok, so now I connected the ov5640 module as per your suggestion and
> > this is what I get with a clean 5.6.2:
> >
> > [    6.618296] imx-media: ov5640 1-003c:0 -> imx6-mipi-csi2:0
> > [    6.644522] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video8
>
> This shows that imx-media completed probe (v4l2 core called its probe
> complete callback).

Yes, correct: /dev/video8 is the mem2mem device.

> > However,  the v4l2video8convert Gstreamer element is not detected:
> >
> > # gst-inspect-1.0 | grep convert
> > video4linux2:  v4l2convert: V4L2 Video Converter
> > audioconvert:  audioconvert: Audio converter
> >
> > If I do the kernel hack I mentioned previously then I can get v4l2video8convert:
> >
> > --- a/drivers/media/v4l2-core/v4l2-async.c
> > +++ b/drivers/media/v4l2-core/v4l2-async.c
> > @@ -213,10 +213,6 @@ v4l2_async_notifier_try_complete(struct
> > v4l2_async_notifier *notifier)
> >          if (!notifier->v4l2_dev)
> >                  return 0;
> >
> > -       /* Is everything ready? */
> > -       if (!v4l2_async_notifier_can_complete(notifier))
> > -               return 0;
> > -
> >          return v4l2_async_notifier_call_complete(notifier);
>
> That's very odd, because v4l2_async_notifier_call_complete() was
> definitely called on the imx-media root notifier from above.

Yes, I agree.

> Is there is another v4l2 driver that is not completing?

Yes, there are some that do not complete, but the mem2mem is getting probed.

Maybe a Gstreamer issue?

Nicolas,

I am using Gstreamer 1.16.2 on i.MX6 and cannot get the
v4l2video8convert to show up.

The mem2mem appears at /dev/video8 and it corresponds to i.MX6
ipu_ic_pp csc/scaler.

Any suggestions?

Thanks

Re: imx6: Cannot register mem2mem

[Fabio Estevam]

On Mon, Apr 6, 2020 at 4:39 PM Fabio Estevam <festevam@gmail.com> wrote:

> Nicolas,
>
> I am using Gstreamer 1.16.2 on i.MX6 and cannot get the
> v4l2video8convert to show up.
>
> The mem2mem appears at /dev/video8 and it corresponds to i.MX6
> ipu_ic_pp csc/scaler.
>
> Any suggestions?

It seems that it shows up with v4l2convert instead of v4l2video8convert
and it works fine:

gst-launch-1.0 videotestsrc ! video/x-raw,width=320,height=240 !
v4l2convert ! video/x-raw,width=640,height=480 ! kmssink

So we are all fine then :-)

Thanks

Re: imx6: Cannot register mem2mem

[Sakari Ailus]

On Mon, Apr 06, 2020 at 11:41:33AM -0300, Fabio Estevam wrote:
> On Mon, Apr 6, 2020 at 10:37 AM Fabio Estevam <festevam@gmail.com> wrote:
> >
> > Hi,
> >
> > I am running kernel 5.6.2 on an imx6qp sabresd, but I cannot get the
> > mem2mem element from imx-media-csc-scaler.c to probe.
> >
> > I don't see imx6_media_probe_complete() getting called, hence
> > imx_media_csc_scaler_device_init() is never called and no mem2mem
> > element is registered.
> >
> > Any ideas as to how to get mem2mem registered on i.MX6?
> 
> It seems that v4l2_async_notifier_can_complete() is always false, so
> v4l2_async_notifier_call_complete() is never called.
> 
> If I change it like this:
> 
> --- a/drivers/media/v4l2-core/v4l2-async.c
> +++ b/drivers/media/v4l2-core/v4l2-async.c
> @@ -213,10 +213,6 @@ v4l2_async_notifier_try_complete(struct
> v4l2_async_notifier *notifier)
>         if (!notifier->v4l2_dev)
>                 return 0;
> 
> -       /* Is everything ready? */
> -       if (!v4l2_async_notifier_can_complete(notifier))
> -               return 0;
> -
>         return v4l2_async_notifier_call_complete(notifier);
>  }
> 
> Then the mem2mem driver can be successfully probed:
> 
> [    4.601350] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video11
> 
> Any suggestions?

There are basically async sub-devices that haven't been yet bound in the
notifier. Once they are, then the device registration will continue.

-- 
Sakari Ailus

Re: imx6: Cannot register mem2mem

[Nicolas Dufresne]

Le lundi 06 avril 2020 à 18:26 -0300, Fabio Estevam a écrit :
> On Mon, Apr 6, 2020 at 4:39 PM Fabio Estevam <festevam@gmail.com> wrote:
> 
> > Nicolas,
> > 
> > I am using Gstreamer 1.16.2 on i.MX6 and cannot get the
> > v4l2video8convert to show up.
> > 
> > The mem2mem appears at /dev/video8 and it corresponds to i.MX6
> > ipu_ic_pp csc/scaler.
> > 
> > Any suggestions?
> 
> It seems that it shows up with v4l2convert instead of v4l2video8convert
> and it works fine:
> 
> gst-launch-1.0 videotestsrc ! video/x-raw,width=320,height=240 !
> v4l2convert ! video/x-raw,width=640,height=480 ! kmssink
> 
> So we are all fine then :-)

Yes, the first to be found "converter", even though the identification
of such converter is a bit bogus, will have the fixed name v4l2convert.
That's the different between 1.14 and 1.16. I've wondering if I should
kept an alias for this case, but we decided not to pollute the registry
with that. The argument was the video8 is not predictable. sometimes
just plugging some UVC camera, or a change in kernel timing will swap
the videoN nodes. Hope this is fine, for simple cases like Exynos and
i.MX6 that should work.

p.s. also, remember the registry is cached, it could not be impossible
that there still some potential cache stalls as tracking the movement
in /dev isn't very reliable.

> 
> Thanks