[PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Javi Merino]

In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
a devicetree overlay, its of_node pointer will be different each time
the overlay is applied.  We are not interested in matching the
pointer, what we want to match is that the path is the one we are
expecting.  Change to use of_node_cmp() so that we continue matching
after the overlay has been removed and reapplied.

Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
Cc: Javier Martinez Canillas <javier@osg.samsung.com>
Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
Signed-off-by: Javi Merino <javi.merino@kernel.org>
---
Hi,

I feel it is a bit of a hack, but I couldn't think of anything better.
I'm ccing devicetree@ and Pantelis because there may be a simpler
solution.

 drivers/media/v4l2-core/v4l2-async.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
index 5bada20..d33a17c 100644
--- a/drivers/media/v4l2-core/v4l2-async.c
+++ b/drivers/media/v4l2-core/v4l2-async.c
@@ -42,7 +42,8 @@ static bool match_devname(struct v4l2_subdev *sd,
 
 static bool match_of(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
 {
-	return sd->of_node == asd->match.of.node;
+	return !of_node_cmp(of_node_full_name(sd->of_node),
+			    of_node_full_name(asd->match.of.node));
 }
 
 static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
-- 
2.1.4

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Javier Martinez Canillas]

Hello Javi,

On 11/23/2016 07:09 AM, Javi Merino wrote:
> In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> a devicetree overlay, its of_node pointer will be different each time
> the overlay is applied.  We are not interested in matching the
> pointer, what we want to match is that the path is the one we are
> expecting.  Change to use of_node_cmp() so that we continue matching
> after the overlay has been removed and reapplied.
>

I'm still not that familiar with DT overlays (and I guess others aren't
either) so I think that including an example of a base tree and overlay
DTS where this is an issue, could make things more clear in the commit.

IIUC, it should be something like this?

-- base tree --

&i2c1 {
	camera: camera@10 {
		reg = <0x10>;
		port {
			cam_ep: endpoint {
				...
			};
		};
	};
};

&media_bridge {
	...
	ports {
		port@0 {
			reg = <0>;
			ep: endpoint {
				remote-endpoint = <&cam_ep>;
			};
		};
	};
};

-- overlay --

/plugin/;
/ {
	...
	fragment@0 {
		target = <&camera>;
		__overlay__ {
			compatible = "foo,bar";
			...
			port {
				cam_ep: endpoint {
					...
				};
			};
		};
	}
}

> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> Signed-off-by: Javi Merino <javi.merino@kernel.org>
> ---
> Hi,
> 
> I feel it is a bit of a hack, but I couldn't think of anything better.
> I'm ccing devicetree@ and Pantelis because there may be a simpler
> solution.
>

I also couldn't think a better way to do this, since IIUC the node's name is
the only thing that doesn't change, and is available at the time the bridge
driver calls v4l2_async_notifier_register() when parsing the base tree.

>  drivers/media/v4l2-core/v4l2-async.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
> index 5bada20..d33a17c 100644
> --- a/drivers/media/v4l2-core/v4l2-async.c
> +++ b/drivers/media/v4l2-core/v4l2-async.c
> @@ -42,7 +42,8 @@ static bool match_devname(struct v4l2_subdev *sd,
>  
>  static bool match_of(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
>  {
> -	return sd->of_node == asd->match.of.node;
> +	return !of_node_cmp(of_node_full_name(sd->of_node),
> +			    of_node_full_name(asd->match.of.node));
>  }
>  
>  static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
> 

Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>

Best regards,
-- 
Javier Martinez Canillas
Open Source Group
Samsung Research America

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Sakari Ailus]

Hi Javi,

On Wed, Nov 23, 2016 at 10:09:57AM +0000, Javi Merino wrote:
> In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> a devicetree overlay, its of_node pointer will be different each time
> the overlay is applied.  We are not interested in matching the
> pointer, what we want to match is that the path is the one we are
> expecting.  Change to use of_node_cmp() so that we continue matching
> after the overlay has been removed and reapplied.
> 
> Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> Signed-off-by: Javi Merino <javi.merino@kernel.org>
> ---
> Hi,
> 
> I feel it is a bit of a hack, but I couldn't think of anything better.
> I'm ccing devicetree@ and Pantelis because there may be a simpler
> solution.

First I have to admit that I'm not an expert when it comes to DT overlays.

That said, my understanding is that the sub-device and the async sub-device
are supposed to point to the exactly same DT node. I wonder if there's
actually anything wrong in the current code.

If the overlay has changed between probing the driver for the async notifier
and the async sub-device, there should be no match here, should there? The
two nodes actually point to a node in a different overlay in that case.

> 
>  drivers/media/v4l2-core/v4l2-async.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
> index 5bada20..d33a17c 100644
> --- a/drivers/media/v4l2-core/v4l2-async.c
> +++ b/drivers/media/v4l2-core/v4l2-async.c
> @@ -42,7 +42,8 @@ static bool match_devname(struct v4l2_subdev *sd,
>  
>  static bool match_of(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
>  {
> -	return sd->of_node == asd->match.of.node;
> +	return !of_node_cmp(of_node_full_name(sd->of_node),
> +			    of_node_full_name(asd->match.of.node));
>  }
>  
>  static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Javi Merino]

On Wed, Nov 23, 2016 at 11:25:39AM -0300, Javier Martinez Canillas wrote:
> Hello Javi,
> 
> On 11/23/2016 07:09 AM, Javi Merino wrote:
> > In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> > a devicetree overlay, its of_node pointer will be different each time
> > the overlay is applied.  We are not interested in matching the
> > pointer, what we want to match is that the path is the one we are
> > expecting.  Change to use of_node_cmp() so that we continue matching
> > after the overlay has been removed and reapplied.
> >
> 
> I'm still not that familiar with DT overlays (and I guess others aren't
> either) so I think that including an example of a base tree and overlay
> DTS where this is an issue, could make things more clear in the commit.
> 
> IIUC, it should be something like this?
> 
> -- base tree --
> 
> &i2c1 {
> 	camera: camera@10 {
> 		reg = <0x10>;
> 		port {
> 			cam_ep: endpoint {
> 				...
> 			};
> 		};
> 	};
> };
> 
> &media_bridge {
> 	...
> 	ports {
> 		port@0 {
> 			reg = <0>;
> 			ep: endpoint {
> 				remote-endpoint = <&cam_ep>;
> 			};
> 		};
> 	};
> };
> 
> -- overlay --
> 
> /plugin/;
> / {
> 	...
> 	fragment@0 {
> 		target = <&camera>;
> 		__overlay__ {
> 			compatible = "foo,bar";
> 			...
> 			port {
> 				cam_ep: endpoint {
> 					...
> 				};
> 			};
> 		};
> 	}
> }

Yes, that's right.  What I have is that the whole camera can be
plugged or unplugged, so the overlay adds/removes the camera node:

/ {
	fragment@0 {
		target-path = "/i2c0";
		__overlay__ {
			my_cam {
				compatible = "foo,bar";
				port {
					camera0: endpoint {
						remote-endpoint = <&vin2a>;
						...
					};
				};
			};
		};
	};

I will add it to the commit message.

> > Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> > Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> > Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Signed-off-by: Javi Merino <javi.merino@kernel.org>
> > ---
> > Hi,
> > 
> > I feel it is a bit of a hack, but I couldn't think of anything better.
> > I'm ccing devicetree@ and Pantelis because there may be a simpler
> > solution.
> >
> 
> I also couldn't think a better way to do this, since IIUC the node's name is
> the only thing that doesn't change, and is available at the time the bridge
> driver calls v4l2_async_notifier_register() when parsing the base tree.
> 
> >  drivers/media/v4l2-core/v4l2-async.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
> > index 5bada20..d33a17c 100644
> > --- a/drivers/media/v4l2-core/v4l2-async.c
> > +++ b/drivers/media/v4l2-core/v4l2-async.c
> > @@ -42,7 +42,8 @@ static bool match_devname(struct v4l2_subdev *sd,
> >  
> >  static bool match_of(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
> >  {
> > -	return sd->of_node == asd->match.of.node;
> > +	return !of_node_cmp(of_node_full_name(sd->of_node),
> > +			    of_node_full_name(asd->match.of.node));
> >  }
> >  
> >  static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
> > 
> 
> Reviewed-by: Javier Martinez Canillas <javier@osg.samsung.com>
> 
> Best regards,
> -- 
> Javier Martinez Canillas
> Open Source Group
> Samsung Research America

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Javi Merino]

On Wed, Nov 23, 2016 at 05:10:42PM +0200, Sakari Ailus wrote:
> Hi Javi,

Hi Sakari,

> On Wed, Nov 23, 2016 at 10:09:57AM +0000, Javi Merino wrote:
> > In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> > a devicetree overlay, its of_node pointer will be different each time
> > the overlay is applied.  We are not interested in matching the
> > pointer, what we want to match is that the path is the one we are
> > expecting.  Change to use of_node_cmp() so that we continue matching
> > after the overlay has been removed and reapplied.
> > 
> > Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> > Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> > Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> > Signed-off-by: Javi Merino <javi.merino@kernel.org>
> > ---
> > Hi,
> > 
> > I feel it is a bit of a hack, but I couldn't think of anything better.
> > I'm ccing devicetree@ and Pantelis because there may be a simpler
> > solution.
> 
> First I have to admit that I'm not an expert when it comes to DT overlays.
> 
> That said, my understanding is that the sub-device and the async sub-device
> are supposed to point to the exactly same DT node. I wonder if there's
> actually anything wrong in the current code.
> 
> If the overlay has changed between probing the driver for the async notifier
> and the async sub-device, there should be no match here, should there? The
> two nodes actually point to a node in a different overlay in that case.

Overlays are parts of the devicetree that can be added and removed.
When the overlay is applied, the camera driver is probed and does
v4l2_async_register_subdev().  However, v4l2_async_belongs() fails.
The problem is with comparing pointers.  I haven't looked at the
implementation of overlays in detail, but what I see is that the
of_node pointer changes when you remove and reapply an overlay (I
guess it's dynamically allocated and when you remove the overlay, it's
freed).

Cheers,
Javi

> > 
> >  drivers/media/v4l2-core/v4l2-async.c | 3 ++-
> >  1 file changed, 2 insertions(+), 1 deletion(-)
> > 
> > diff --git a/drivers/media/v4l2-core/v4l2-async.c b/drivers/media/v4l2-core/v4l2-async.c
> > index 5bada20..d33a17c 100644
> > --- a/drivers/media/v4l2-core/v4l2-async.c
> > +++ b/drivers/media/v4l2-core/v4l2-async.c
> > @@ -42,7 +42,8 @@ static bool match_devname(struct v4l2_subdev *sd,
> >  
> >  static bool match_of(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
> >  {
> > -	return sd->of_node == asd->match.of.node;
> > +	return !of_node_cmp(of_node_full_name(sd->of_node),
> > +			    of_node_full_name(asd->match.of.node));
> >  }
> >  
> >  static bool match_custom(struct v4l2_subdev *sd, struct v4l2_async_subdev *asd)
> 
> -- 
> Kind regards,
> 
> Sakari Ailus
> e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Sakari Ailus]

Hi Javi,

On Wed, Nov 23, 2016 at 04:15:11PM +0000, Javi Merino wrote:
> On Wed, Nov 23, 2016 at 05:10:42PM +0200, Sakari Ailus wrote:
> > Hi Javi,
> 
> Hi Sakari,
> 
> > On Wed, Nov 23, 2016 at 10:09:57AM +0000, Javi Merino wrote:
> > > In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> > > a devicetree overlay, its of_node pointer will be different each time
> > > the overlay is applied.  We are not interested in matching the
> > > pointer, what we want to match is that the path is the one we are
> > > expecting.  Change to use of_node_cmp() so that we continue matching
> > > after the overlay has been removed and reapplied.
> > > 
> > > Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> > > Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> > > Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > Signed-off-by: Javi Merino <javi.merino@kernel.org>
> > > ---
> > > Hi,
> > > 
> > > I feel it is a bit of a hack, but I couldn't think of anything better.
> > > I'm ccing devicetree@ and Pantelis because there may be a simpler
> > > solution.
> > 
> > First I have to admit that I'm not an expert when it comes to DT overlays.
> > 
> > That said, my understanding is that the sub-device and the async sub-device
> > are supposed to point to the exactly same DT node. I wonder if there's
> > actually anything wrong in the current code.
> > 
> > If the overlay has changed between probing the driver for the async notifier
> > and the async sub-device, there should be no match here, should there? The
> > two nodes actually point to a node in a different overlay in that case.
> 
> Overlays are parts of the devicetree that can be added and removed.
> When the overlay is applied, the camera driver is probed and does
> v4l2_async_register_subdev().  However, v4l2_async_belongs() fails.
> The problem is with comparing pointers.  I haven't looked at the
> implementation of overlays in detail, but what I see is that the
> of_node pointer changes when you remove and reapply an overlay (I
> guess it's dynamically allocated and when you remove the overlay, it's
> freed).

The concern here which we were discussing was whether the overlay should be
relied on having compliant configuration compared to the part which was not
part of the overlay.

As external components are involved, quite possibly also the ISP DT node
will require changes, not just the image source (TV tuner, camera sensor
etc.). This could be because of number of CSI-2 lanes or parallel bus width,
for instance.

I'd also be interested in having an actual driver implement support for
removing and adding a DT overlay first so we'd see how this would actually
work. We need both in order to be able to actually remove and add DT
overlays _without_ unbinding the ISP driver. Otherwise it should already
work in the current codebase.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH] v4l: async: make v4l2 coexists with devicetree nodes in a dt overlay

[Javi Merino]

On Fri, Nov 25, 2016 at 10:21:21AM +0200, Sakari Ailus wrote:
Hi Sakari,

> On Wed, Nov 23, 2016 at 04:15:11PM +0000, Javi Merino wrote:
> > On Wed, Nov 23, 2016 at 05:10:42PM +0200, Sakari Ailus wrote:
> > > Hi Javi,
> > 
> > Hi Sakari,
> > 
> > > On Wed, Nov 23, 2016 at 10:09:57AM +0000, Javi Merino wrote:
> > > > In asd's configured with V4L2_ASYNC_MATCH_OF, if the v4l2 subdev is in
> > > > a devicetree overlay, its of_node pointer will be different each time
> > > > the overlay is applied.  We are not interested in matching the
> > > > pointer, what we want to match is that the path is the one we are
> > > > expecting.  Change to use of_node_cmp() so that we continue matching
> > > > after the overlay has been removed and reapplied.
> > > > 
> > > > Cc: Mauro Carvalho Chehab <mchehab@kernel.org>
> > > > Cc: Javier Martinez Canillas <javier@osg.samsung.com>
> > > > Cc: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > > Signed-off-by: Javi Merino <javi.merino@kernel.org>
> > > > ---
> > > > Hi,
> > > > 
> > > > I feel it is a bit of a hack, but I couldn't think of anything better.
> > > > I'm ccing devicetree@ and Pantelis because there may be a simpler
> > > > solution.
> > > 
> > > First I have to admit that I'm not an expert when it comes to DT overlays.
> > > 
> > > That said, my understanding is that the sub-device and the async sub-device
> > > are supposed to point to the exactly same DT node. I wonder if there's
> > > actually anything wrong in the current code.
> > > 
> > > If the overlay has changed between probing the driver for the async notifier
> > > and the async sub-device, there should be no match here, should there? The
> > > two nodes actually point to a node in a different overlay in that case.
> > 
> > Overlays are parts of the devicetree that can be added and removed.
> > When the overlay is applied, the camera driver is probed and does
> > v4l2_async_register_subdev().  However, v4l2_async_belongs() fails.
> > The problem is with comparing pointers.  I haven't looked at the
> > implementation of overlays in detail, but what I see is that the
> > of_node pointer changes when you remove and reapply an overlay (I
> > guess it's dynamically allocated and when you remove the overlay, it's
> > freed).
> 
> The concern here which we were discussing was whether the overlay should be
> relied on having compliant configuration compared to the part which was not
> part of the overlay.
> 
> As external components are involved, quite possibly also the ISP DT node
> will require changes, not just the image source (TV tuner, camera sensor
> etc.). This could be because of number of CSI-2 lanes or parallel bus width,
> for instance.
> 
> I'd also be interested in having an actual driver implement support for
> removing and adding a DT overlay first so we'd see how this would actually
> work. We need both in order to be able to actually remove and add DT
> overlays _without_ unbinding the ISP driver. Otherwise it should already
> work in the current codebase.

Unfortunately, the driver I'm working on is not upstream and I can't
submit it to mainline.  This patch fixes the issue for me, so I
thought it could be useful fix for the kernel.

Cheers,
Javi