All of lore.kernel.org
 help / color / mirror / Atom feed
* Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?
@ 2017-02-10  9:34 Thomas Axelsson
  2017-02-12 22:16 ` Sakari Ailus
  0 siblings, 1 reply; 3+ messages in thread
From: Thomas Axelsson @ 2017-02-10  9:34 UTC (permalink / raw)
  To: linux-media

Hi,

I have a TI AM571x CPU, and I'm trying to add support for multiple MIPI CSI-2 
virtual channels per PHY (port) to the ti-vpe/cal.c driver (CAMSS/CAL peripheral,  
ch. 8 in Datasheet [1]). This CPU can have more contexts (virtual channels) per PHY 
than what it has DMA handlers. Each PHY may have up to 8 contexts, and there are 2 
PHYs, but there are only 8 DMA channels in total. It is not required to use DMA to 
receive data from the context.
 
Since it will be very useful to specify which contexts will use DMA and which will 
not, the proper place to do this seems to be the device tree.
 
This becomes rather messy though, since it needs to be specified in the device tree 
node pointed to by the remote-endpoint field - yet, it's decided by the capabilities 
of the master component (in this case the CAL), so the remote-endpoint is a weird 
place to put it.
 
I have made an example [2] using the Device Tree example in 
Documentation/devicetree/bindings/media/ti-cal.txt (my own comments).
In the ar0330_1 endpoint, I have:
* Put multiple virtual channels in "reg", as in 
  Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt,
* Added "dma-write" for specifying which virtual channels should get written 
  directly to memory through DMA,
* Added "vip" just to show that a Virtual Channel can go somewhere else than 
  through DMA write.
* Added "pix-proc" to show that pixel processing might be applied to some of the 
  Virtual Channels.
 
What is your advice on how to properly move forward with adding support like this?

Thank you in advance.

Best regards,
Thomas Axelsson
 

[1] http://www.ti.com/lit/gpn/am5716
 
[2]
--------------------------------------------------
cal: cal@4845b000 {
    compatible = "ti,dra72-cal";
    ti,hwmods = "cal";
 
    /* snip */
 
    ports {
        #address-cells = <1>;
        #size-cells = <0>;
        csi2_0: port@0 {
            reg = <0>;                         /* PHY index, must match port index */
            status = "okay";                   /* Enable */
            endpoint {
                slave-mode;
                remote-endpoint = <&ar0330_1>;
            };
        };
        csi2_1: port@1 {
            reg = <1>;                         /* PHY Index */
        };
    };
};
 
i2c5: i2c@4807c000 {
    ar0330@10 {
        compatible = "ti,ar0330";
        reg = <0x10>;
        port {
            #address-cells = <1>;
            #size-cells = <0>;
            ar0330_1: endpoint {
                reg = <0 1 2>;                 /* Virtual Channels */
                dma-write = <0 2>;             /* Virtual Channels that will use 
                                                  Write DMA */
                vip = <1>;                     /* Virtual Channel to send on to 
                                                  Video Input Port */
                pix-proc = <2>;                /* Virtual channels to apply pixel
                                                  processing on */
                clock-lanes = <1>;             /* Clock lane indices */
                data-lanes = <0 2 3 4>;        /* Data lane indices */
                remote-endpoint = <&csi2_0>;
            };
        };
    };
};
--------------------------------------------------

^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?
  2017-02-10  9:34 Device Tree formatting for multiple virtual channels in ti-vpe/cal driver? Thomas Axelsson
@ 2017-02-12 22:16 ` Sakari Ailus
  2017-02-13  9:33   ` Thomas Axelsson
  0 siblings, 1 reply; 3+ messages in thread
From: Sakari Ailus @ 2017-02-12 22:16 UTC (permalink / raw)
  To: Thomas Axelsson; +Cc: linux-media

Hi Thomas,

On Fri, Feb 10, 2017 at 09:34:46AM +0000, Thomas Axelsson wrote:
> Hi,
> 
> I have a TI AM571x CPU, and I'm trying to add support for multiple MIPI
> CSI-2 virtual channels per PHY (port) to the ti-vpe/cal.c driver
> (CAMSS/CAL peripheral, ch. 8 in Datasheet [1]). This CPU can have more
> contexts (virtual channels) per PHY than what it has DMA handlers. Each
> PHY may have up to 8 contexts, and there are 2 PHYs, but there are only 8
> DMA channels in total. It is not required to use DMA to receive data from
> the context.

Is there a use case for receiving more than eight streams concurrently? I
have to admit that this does appear quite exotic if 8 would not suffice. How
does the data end up to the system memory if there's no DMA? PIO...?

What are the limitations otherwise --- how many PHYs can be used
simultaneously? Are the 8 DMAs shared among all?

>  
> Since it will be very useful to specify which contexts will use DMA and which will 
> not, the proper place to do this seems to be the device tree.
>  
> This becomes rather messy though, since it needs to be specified in the device tree 
> node pointed to by the remote-endpoint field - yet, it's decided by the capabilities 
> of the master component (in this case the CAL), so the remote-endpoint is a weird 
> place to put it.
>  
> I have made an example [2] using the Device Tree example in 
> Documentation/devicetree/bindings/media/ti-cal.txt (my own comments).
> In the ar0330_1 endpoint, I have:
> * Put multiple virtual channels in "reg", as in 
>   Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt,
> * Added "dma-write" for specifying which virtual channels should get written 
>   directly to memory through DMA,
> * Added "vip" just to show that a Virtual Channel can go somewhere else than 
>   through DMA write.
> * Added "pix-proc" to show that pixel processing might be applied to some of the 
>   Virtual Channels.
>  
> What is your advice on how to properly move forward with adding support like this?
> 
> Thank you in advance.
> 
> Best regards,
> Thomas Axelsson
>  
> 
> [1] http://www.ti.com/lit/gpn/am5716
>  
> [2]
> --------------------------------------------------
> cal: cal@4845b000 {
>     compatible = "ti,dra72-cal";
>     ti,hwmods = "cal";
>  
>     /* snip */
>  
>     ports {
>         #address-cells = <1>;
>         #size-cells = <0>;
>         csi2_0: port@0 {
>             reg = <0>;                         /* PHY index, must match port index */
>             status = "okay";                   /* Enable */
>             endpoint {
>                 slave-mode;
>                 remote-endpoint = <&ar0330_1>;
>             };
>         };
>         csi2_1: port@1 {
>             reg = <1>;                         /* PHY Index */
>         };
>     };
> };
>  
> i2c5: i2c@4807c000 {
>     ar0330@10 {
>         compatible = "ti,ar0330";
>         reg = <0x10>;
>         port {
>             #address-cells = <1>;
>             #size-cells = <0>;
>             ar0330_1: endpoint {
>                 reg = <0 1 2>;                 /* Virtual Channels */
>                 dma-write = <0 2>;             /* Virtual Channels that will use 
>                                                   Write DMA */
>                 vip = <1>;                     /* Virtual Channel to send on to 
>                                                   Video Input Port */
>                 pix-proc = <2>;                /* Virtual channels to apply pixel
>                                                   processing on */
>                 clock-lanes = <1>;             /* Clock lane indices */
>                 data-lanes = <0 2 3 4>;        /* Data lane indices */
>                 remote-endpoint = <&csi2_0>;
>             };
>         };
>     };
> };

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

^ permalink raw reply	[flat|nested] 3+ messages in thread

* RE: Device Tree formatting for multiple virtual channels in ti-vpe/cal driver?
  2017-02-12 22:16 ` Sakari Ailus
@ 2017-02-13  9:33   ` Thomas Axelsson
  0 siblings, 0 replies; 3+ messages in thread
From: Thomas Axelsson @ 2017-02-13  9:33 UTC (permalink / raw)
  To: Sakari Ailus; +Cc: linux-media

Hi Sakari,

> -----Original Message-----
> From: Sakari Ailus [mailto:sakari.ailus@iki.fi]
> Sent: den 12 februari 2017 23:17
> To: Thomas Axelsson <Thomas.Axelsson@cybercom.com>
> Cc: linux-media@vger.kernel.org
> Subject: Re: Device Tree formatting for multiple virtual channels in ti-vpe/cal
> driver?
> 
> Hi Thomas,
> 
> On Fri, Feb 10, 2017 at 09:34:46AM +0000, Thomas Axelsson wrote:
> > Hi,
> >
> > I have a TI AM571x CPU, and I'm trying to add support for multiple
> > MIPI
> > CSI-2 virtual channels per PHY (port) to the ti-vpe/cal.c driver
> > (CAMSS/CAL peripheral, ch. 8 in Datasheet [1]). This CPU can have more
> > contexts (virtual channels) per PHY than what it has DMA handlers.
> > Each PHY may have up to 8 contexts, and there are 2 PHYs, but there
> > are only 8 DMA channels in total. It is not required to use DMA to
> > receive data from the context.
> 
> Is there a use case for receiving more than eight streams concurrently? I
> have to admit that this does appear quite exotic if 8 would not suffice. How
> does the data end up to the system memory if there's no DMA? PIO...?
> 
> What are the limitations otherwise --- how many PHYs can be used
> simultaneously? Are the 8 DMAs shared among all?
> 

I cannot envision such a use case at the moment, but one can choose to not 
send a video stream to the DMAs, and instead send it only to the VIP (Video 
Input Port), which has its own DMA. However, I will never use all DMAs, so 
just handing them out to each context will suffice in my case, and I do not 
intend to use the VIP. I will probably skip adding Device Tree support for 
specifying specific DMAs and VIP.

Both PHYs can be used at the same time. The 8 DMAs are shared by the two 
PHYs.

> >
> > Since it will be very useful to specify which contexts will use DMA
> > and which will not, the proper place to do this seems to be the device tree.
> >
> > This becomes rather messy though, since it needs to be specified in
> > the device tree node pointed to by the remote-endpoint field - yet,
> > it's decided by the capabilities of the master component (in this case
> > the CAL), so the remote-endpoint is a weird place to put it.
> >
> > I have made an example [2] using the Device Tree example in
> > Documentation/devicetree/bindings/media/ti-cal.txt (my own comments).
> > In the ar0330_1 endpoint, I have:
> > * Put multiple virtual channels in "reg", as in
> >   Documentation/devicetree/bindings/mipi/dsi/mipi-dsi-bus.txt,
> > * Added "dma-write" for specifying which virtual channels should get
> written
> >   directly to memory through DMA,
> > * Added "vip" just to show that a Virtual Channel can go somewhere else
> than
> >   through DMA write.
> > * Added "pix-proc" to show that pixel processing might be applied to some
> of the
> >   Virtual Channels.
> >
> > What is your advice on how to properly move forward with adding support
> like this?
> >
> > Thank you in advance.
> >
> > Best regards,
> > Thomas Axelsson
> >
> >
> > [1] http://www.ti.com/lit/gpn/am5716
> >
> > [2]
> > --------------------------------------------------
> > cal: cal@4845b000 {
> >     compatible = "ti,dra72-cal";
> >     ti,hwmods = "cal";
> >
> >     /* snip */
> >
> >     ports {
> >         #address-cells = <1>;
> >         #size-cells = <0>;
> >         csi2_0: port@0 {
> >             reg = <0>;                         /* PHY index, must match port index */
> >             status = "okay";                   /* Enable */
> >             endpoint {
> >                 slave-mode;
> >                 remote-endpoint = <&ar0330_1>;
> >             };
> >         };
> >         csi2_1: port@1 {
> >             reg = <1>;                         /* PHY Index */
> >         };
> >     };
> > };
> >
> > i2c5: i2c@4807c000 {
> >     ar0330@10 {
> >         compatible = "ti,ar0330";
> >         reg = <0x10>;
> >         port {
> >             #address-cells = <1>;
> >             #size-cells = <0>;
> >             ar0330_1: endpoint {
> >                 reg = <0 1 2>;                 /* Virtual Channels */
> >                 dma-write = <0 2>;             /* Virtual Channels that will use
> >                                                   Write DMA */
> >                 vip = <1>;                     /* Virtual Channel to send on to
> >                                                   Video Input Port */
> >                 pix-proc = <2>;                /* Virtual channels to apply pixel
> >                                                   processing on */
> >                 clock-lanes = <1>;             /* Clock lane indices */
> >                 data-lanes = <0 2 3 4>;        /* Data lane indices */
> >                 remote-endpoint = <&csi2_0>;
> >             };
> >         };
> >     };
> > };
> 
> --
> Kind regards,
> 
> Sakari Ailus
> e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk


I have dug some more in V4L2, and realized that simply expanding the remote 
endpoint "reg", to contain an array of values representing virtual 
channels, will prevent identifying virtual channels on a higher level (both 
ioctl and v4l2_subdev owner). This makes it impossible to configure virtual 
channel level options, like video format. I need to be able to configure a 
different video format for each virtual channel.

The following sequence diagram illustrates the problem:

         CAL             sensor chip           
          |                   |
          |------- probe ---->|
          |                   |
          |<------ subdev ----|
          |                   |
   map subdev to PHY          |
          |                   |
  4 VCs from remote "reg"     |
   - map to contexts          |
          |                   |
          :                   :
          |------- get_fmt -->|   Which VC!?
          |                   |
          |<------ err? ------|
          |                   |                userspace
          |                   |                   |
          |                   |<- ioctl: get_fmt -|   Which VC!?
          |                   |                   |
          |                   |-------- err? ---->|
          |                   |                   |


Is this approached aligned with the V4L2 design? How will I handle the 
above scenario in such case?


I have started to think that I will need to move virtual channel mapping up 
to "endpoint" level in the Device Tree, i.e. only have one value in the 
remote endpoint "reg", but have an endpoint for each virtual channel (both 
"local" and remote endpoint, since they map 1:1). In that case, each 
virtual channel will get its unique v4l2_subdev, with which I can separate 
the virtual channels within the driver and in the ioctls.

By changing the endpoints to map to virtual channels, the endpoints will no 
longer map to PHYs, and the lane configuration does no longer make sense to 
have in the endpoint configurations, but rather on the port level. I'm 
feeling that I will start to leave the V4L2 Device Tree design.

Any thoughts?

Best Regards,
Thomas Axelsson

^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2017-02-13  9:33 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-02-10  9:34 Device Tree formatting for multiple virtual channels in ti-vpe/cal driver? Thomas Axelsson
2017-02-12 22:16 ` Sakari Ailus
2017-02-13  9:33   ` Thomas Axelsson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.