* [PATCH/RFC 0/1] v4l: Add support for binary controls @ 2010-03-29 9:53 Kamil Debski 2010-03-29 9:53 ` [PATCH/RFC 1/1] " Kamil Debski 2010-03-30 6:41 ` [PATCH/RFC 0/1] " Hans Verkuil 0 siblings, 2 replies; 10+ messages in thread From: Kamil Debski @ 2010-03-29 9:53 UTC (permalink / raw) To: linux-media; +Cc: p.osciak, k.debski, kyungmin.park Hello, This patch introduces new type of v4l2 control - the binary control. It will be useful for exchanging raw binary data between the user space and the driver/hardware. The patch is pretty small – basically it adds a new control type. 1. Reasons to include this new type - Some devices require data which are not part of the stream, but there are necessary for the device to work e.g. coefficients for transformation matrices. - String control is not suitable as it suggests that the data is a null terminated string. This might be important when printing debug information - one might output strings as they are and binary data in hex. 2. How does the binary control work The binary control has been based on the string control. The principle of use is the same. It uses v4l2_ext_control structure to pass the pointer and size of the data. It is left for the driver to call the copy_from_user/ copy_to_user function to copy the data. 3. About the patch The patch is pretty small – it basically adds a new control type. Best wishes, -- Kamil Debski Linux Platform Group Samsung Poland R&D Center ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH/RFC 1/1] v4l: Add support for binary controls 2010-03-29 9:53 [PATCH/RFC 0/1] v4l: Add support for binary controls Kamil Debski @ 2010-03-29 9:53 ` Kamil Debski 2010-03-30 6:41 ` [PATCH/RFC 0/1] " Hans Verkuil 1 sibling, 0 replies; 10+ messages in thread From: Kamil Debski @ 2010-03-29 9:53 UTC (permalink / raw) To: linux-media; +Cc: p.osciak, k.debski, kyungmin.park This control provides means to exchange raw binary data between the user space and the driver. Signed-off-by: Kamil Debski <k.debski@samsung.com> Reviewed-by: Kyungmin Park <kyungmin.park@samsung.com> --- Documentation/DocBook/v4l/compat.xml | 9 ++++++++ Documentation/DocBook/v4l/videodev2.h.xml | 2 + Documentation/DocBook/v4l/vidioc-g-ext-ctrls.xml | 6 +++++ Documentation/DocBook/v4l/vidioc-queryctrl.xml | 23 ++++++++++++++++----- drivers/media/video/v4l2-common.c | 2 + include/linux/videodev2.h | 2 + 6 files changed, 38 insertions(+), 6 deletions(-) diff --git a/Documentation/DocBook/v4l/compat.xml b/Documentation/DocBook/v4l/compat.xml index b9dbdf9..83b8b64 100644 --- a/Documentation/DocBook/v4l/compat.xml +++ b/Documentation/DocBook/v4l/compat.xml @@ -2333,6 +2333,15 @@ more information.</para> </orderedlist> </section> </section> + <section> + <title>V4L2 in Linux 2.6.35</title> + <orderedlist> + <listitem> + <para>Added support for binary data controls via new type <constant>V4L2_CTRL_TYPE_BINARY</constant>.</para> + </listitem> + </orderedlist> + </section> + </section> <section id="other"> <title>Relation of V4L2 to other Linux multimedia APIs</title> diff --git a/Documentation/DocBook/v4l/videodev2.h.xml b/Documentation/DocBook/v4l/videodev2.h.xml index 0683259..c552134 100644 --- a/Documentation/DocBook/v4l/videodev2.h.xml +++ b/Documentation/DocBook/v4l/videodev2.h.xml @@ -169,6 +169,7 @@ enum <link linkend="v4l2-ctrl-type">v4l2_ctrl_type</link> { V4L2_CTRL_TYPE_INTEGER64 = 5, V4L2_CTRL_TYPE_CTRL_CLASS = 6, V4L2_CTRL_TYPE_STRING = 7, + V4L2_CTRL_TYPE_BINARY = 8, }; enum <link linkend="v4l2-tuner-type">v4l2_tuner_type</link> { @@ -913,6 +914,7 @@ struct <link linkend="v4l2-ext-control">v4l2_ext_control</link> { __s32 value; __s64 value64; char *string; + unsigned char *blob; }; } __attribute__ ((packed)); diff --git a/Documentation/DocBook/v4l/vidioc-g-ext-ctrls.xml b/Documentation/DocBook/v4l/vidioc-g-ext-ctrls.xml index 3aa7f8f..ab4f1c9 100644 --- a/Documentation/DocBook/v4l/vidioc-g-ext-ctrls.xml +++ b/Documentation/DocBook/v4l/vidioc-g-ext-ctrls.xml @@ -170,6 +170,12 @@ applications must set the array to zero.</entry> <entry><structfield>string</structfield></entry> <entry>A pointer to a string.</entry> </row> + <row> + <entry></entry> + <entry>unsigned char *</entry> + <entry><structfield>blob</structfield></entry> + <entry>A pointer to a blob.</entry> + </row> </tbody> </tgroup> </table> diff --git a/Documentation/DocBook/v4l/vidioc-queryctrl.xml b/Documentation/DocBook/v4l/vidioc-queryctrl.xml index 4876ff1..d1cd5ee 100644 --- a/Documentation/DocBook/v4l/vidioc-queryctrl.xml +++ b/Documentation/DocBook/v4l/vidioc-queryctrl.xml @@ -142,9 +142,10 @@ bound for <constant>V4L2_CTRL_TYPE_INTEGER</constant> controls and the lowest valid index (always 0) for <constant>V4L2_CTRL_TYPE_MENU</constant> controls. For <constant>V4L2_CTRL_TYPE_STRING</constant> controls the minimum value gives the minimum length of the string. This length <emphasis>does not include the terminating -zero</emphasis>. It may not be valid for any other type of control, including -<constant>V4L2_CTRL_TYPE_INTEGER64</constant> controls. Note that this is a -signed value.</entry> +zero</emphasis>. For <constant>V4L2_CTRL_TYPE_BINARY</constant> controls this value +gives the minimum length of the binary data. It may not be valid for any other type of + control, including <constant>V4L2_CTRL_TYPE_INTEGER64</constant> controls. Note that + this is a signed value.</entry> </row> <row> <entry>__s32</entry> @@ -155,7 +156,8 @@ highest valid index for <constant>V4L2_CTRL_TYPE_MENU</constant> controls. For <constant>V4L2_CTRL_TYPE_STRING</constant> controls the maximum value gives the maximum length of the string. This length <emphasis>does not include the terminating -zero</emphasis>. It may not be valid for any other type of control, including +zero</emphasis>. For <constant>V4L2_CTRL_TYPE_BINARY</constant> controls this value +gives the maximum length of the binary data. It may not be valid for any other type of control, including <constant>V4L2_CTRL_TYPE_INTEGER64</constant> controls. Note that this is a signed value.</entry> </row> @@ -166,8 +168,8 @@ signed value.</entry> <constant>V4L2_CTRL_TYPE_INTEGER</constant> controls. For <constant>V4L2_CTRL_TYPE_STRING</constant> controls this field refers to the string length that has to be a multiple of this step size. -It may not be valid for any other type of control, including -<constant>V4L2_CTRL_TYPE_INTEGER64</constant> +For <constant>V4L2_CTRL_TYPE_BINARY</constant> controls the size of the binary data has to be a multiple of this step size. It may not be valid for any other type of +control, including <constant>V4L2_CTRL_TYPE_INTEGER64</constant> controls.</para><para>Generally drivers should not scale hardware control values. It may be necessary for example when the <structfield>name</structfield> or <structfield>id</structfield> imply @@ -319,6 +321,15 @@ Which character encoding is used will depend on the string control itself and should be part of the control documentation.</entry> </row> <row> + <entry><constant>V4L2_CTRL_TYPE_BINARY</constant></entry> + <entry>≥ 0</entry> + <entry>≥ 1</entry> + <entry>≥ 0</entry> + <entry>The minimum and maximum of the binary data length. The step size +means that the length must be (minimum + N * step) characters long for +N ≥ 0. </entry> + </row> + <row> <entry><constant>V4L2_CTRL_TYPE_CTRL_CLASS</constant></entry> <entry>n/a</entry> <entry>n/a</entry> diff --git a/drivers/media/video/v4l2-common.c b/drivers/media/video/v4l2-common.c index 36b5cb8..cbd770e 100644 --- a/drivers/media/video/v4l2-common.c +++ b/drivers/media/video/v4l2-common.c @@ -158,6 +158,8 @@ int v4l2_ctrl_check(struct v4l2_ext_control *ctrl, struct v4l2_queryctrl *qctrl, return -EBUSY; if (qctrl->type == V4L2_CTRL_TYPE_STRING) return 0; + if (qctrl->type == V4L2_CTRL_TYPE_BINARY) + return 0; if (qctrl->type == V4L2_CTRL_TYPE_BUTTON || qctrl->type == V4L2_CTRL_TYPE_INTEGER64 || qctrl->type == V4L2_CTRL_TYPE_CTRL_CLASS) diff --git a/include/linux/videodev2.h b/include/linux/videodev2.h index 3793d16..5a62c9a 100644 --- a/include/linux/videodev2.h +++ b/include/linux/videodev2.h @@ -168,6 +168,7 @@ enum v4l2_ctrl_type { V4L2_CTRL_TYPE_INTEGER64 = 5, V4L2_CTRL_TYPE_CTRL_CLASS = 6, V4L2_CTRL_TYPE_STRING = 7, + V4L2_CTRL_TYPE_BINARY = 8, }; enum v4l2_tuner_type { @@ -918,6 +919,7 @@ struct v4l2_ext_control { __s32 value; __s64 value64; char *string; + unsigned char *blob; }; } __attribute__ ((packed)); -- 1.6.3.3 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-29 9:53 [PATCH/RFC 0/1] v4l: Add support for binary controls Kamil Debski 2010-03-29 9:53 ` [PATCH/RFC 1/1] " Kamil Debski @ 2010-03-30 6:41 ` Hans Verkuil 2010-03-30 8:57 ` Laurent Pinchart 2010-03-30 12:26 ` Kamil Debski 1 sibling, 2 replies; 10+ messages in thread From: Hans Verkuil @ 2010-03-30 6:41 UTC (permalink / raw) To: Kamil Debski; +Cc: linux-media, p.osciak, kyungmin.park Hi Kamil! On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > > Hello, > > This patch introduces new type of v4l2 control - the binary control. It > will be useful for exchanging raw binary data between the user space and > the driver/hardware. > > The patch is pretty small – basically it adds a new control type. > > 1. Reasons to include this new type > - Some devices require data which are not part of the stream, but there > are necessary for the device to work e.g. coefficients for transformation > matrices. > - String control is not suitable as it suggests that the data is a null > terminated string. This might be important when printing debug information - > one might output strings as they are and binary data in hex. > > 2. How does the binary control work > The binary control has been based on the string control. The principle of > use is the same. It uses v4l2_ext_control structure to pass the pointer and > size of the data. It is left for the driver to call the copy_from_user/ > copy_to_user function to copy the data. > > 3. About the patch > The patch is pretty small – it basically adds a new control type. > > Best wishes, > I don't think this is a good idea. Controls are not really meant to be used as an ioctl replacement. Controls can be used to control the hardware via a GUI (e.g. qv4l2). Obviously, this will fail for binary controls. Controls can also be used in cases where it is not known up front which controls are needed. This typically happens for bridge drivers that can use numerous combinations of i2c sub-devices. Each subdev can have its own controls. There is a grey area where you want to give the application access to low-level parameters but without showing them to the end-user. This is currently not possible, but it will be once the control framework is finished and once we have the possibility to create device nodes for subdevs. But what you want is to basically pass whole structs as a control. That's something that ioctls where invented for. Especially once we have subdev nodes this shouldn't be a problem. Just the fact that it is easy to implement doesn't mean it should be done :-) Do you have specific use-cases for your proposed binary control? Regards, Hans -- Hans Verkuil - video4linux developer - sponsored by TANDBERG ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 6:41 ` [PATCH/RFC 0/1] " Hans Verkuil @ 2010-03-30 8:57 ` Laurent Pinchart 2010-03-30 11:43 ` Andy Walls 2010-03-30 12:26 ` Kamil Debski 1 sibling, 1 reply; 10+ messages in thread From: Laurent Pinchart @ 2010-03-30 8:57 UTC (permalink / raw) To: Hans Verkuil; +Cc: Kamil Debski, linux-media, p.osciak, kyungmin.park Hi Hans, On Tuesday 30 March 2010 08:41:47 Hans Verkuil wrote: > On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > > Hello, > > > > This patch introduces new type of v4l2 control - the binary control. It > > will be useful for exchanging raw binary data between the user space and > > the driver/hardware. > > > > The patch is pretty small – basically it adds a new control type. > > > > 1. Reasons to include this new type > > - Some devices require data which are not part of the stream, but there > > are necessary for the device to work e.g. coefficients for transformation > > matrices. > > - String control is not suitable as it suggests that the data is a null > > terminated string. This might be important when printing debug > > information - one might output strings as they are and binary data in > > hex. > > > > 2. How does the binary control work > > The binary control has been based on the string control. The principle of > > use is the same. It uses v4l2_ext_control structure to pass the pointer > > and size of the data. It is left for the driver to call the > > copy_from_user/ copy_to_user function to copy the data. > > > > 3. About the patch > > The patch is pretty small – it basically adds a new control type. > > > > Best wishes, > > I don't think this is a good idea. Controls are not really meant to be used > as an ioctl replacement. > > Controls can be used to control the hardware via a GUI (e.g. qv4l2). > Obviously, this will fail for binary controls. Controls can also be used > in cases where it is not known up front which controls are needed. This > typically happens for bridge drivers that can use numerous combinations of > i2c sub-devices. Each subdev can have its own controls. > > There is a grey area where you want to give the application access to > low-level parameters but without showing them to the end-user. This is > currently not possible, but it will be once the control framework is > finished and once we have the possibility to create device nodes for > subdevs. > > But what you want is to basically pass whole structs as a control. That's > something that ioctls where invented for. Especially once we have subdev > nodes this shouldn't be a problem. > > Just the fact that it is easy to implement doesn't mean it should be done > :-) > > Do you have specific use-cases for your proposed binary control? As discussed yesterday, here are a few use cases for the OMAP3 ISP driver. - white balance matrix - gamma correction tables In both cases, the driver needs an array (possible 2 dimensional) of values to configure the hardware. This can obviously be done using private ioctls, but what makes the red&blue white balance gains different from the white balance matrix ? Why should the first be controls and the later not ? -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 8:57 ` Laurent Pinchart @ 2010-03-30 11:43 ` Andy Walls 0 siblings, 0 replies; 10+ messages in thread From: Andy Walls @ 2010-03-30 11:43 UTC (permalink / raw) To: Laurent Pinchart Cc: Hans Verkuil, Kamil Debski, linux-media, p.osciak, kyungmin.park On Tue, 2010-03-30 at 10:57 +0200, Laurent Pinchart wrote: > Hi Hans, > > On Tuesday 30 March 2010 08:41:47 Hans Verkuil wrote: > > On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > > > Hello, > > > > > > This patch introduces new type of v4l2 control - the binary control. It > > > will be useful for exchanging raw binary data between the user space and > > > the driver/hardware. > > > > > > The patch is pretty small – basically it adds a new control type. > > > > > > 1. Reasons to include this new type > > > - Some devices require data which are not part of the stream, but there > > > are necessary for the device to work e.g. coefficients for transformation > > > matrices. > > > - String control is not suitable as it suggests that the data is a null > > > terminated string. This might be important when printing debug > > > information - one might output strings as they are and binary data in > > > hex. > > > > > > 2. How does the binary control work > > > The binary control has been based on the string control. The principle of > > > use is the same. It uses v4l2_ext_control structure to pass the pointer > > > and size of the data. It is left for the driver to call the > > > copy_from_user/ copy_to_user function to copy the data. > > > > > > 3. About the patch > > > The patch is pretty small – it basically adds a new control type. > > > > > > Best wishes, > > > > I don't think this is a good idea. Controls are not really meant to be used > > as an ioctl replacement. > > > > Controls can be used to control the hardware via a GUI (e.g. qv4l2). > > Obviously, this will fail for binary controls. Controls can also be used > > in cases where it is not known up front which controls are needed. This > > typically happens for bridge drivers that can use numerous combinations of > > i2c sub-devices. Each subdev can have its own controls. > > > > There is a grey area where you want to give the application access to > > low-level parameters but without showing them to the end-user. This is > > currently not possible, but it will be once the control framework is > > finished and once we have the possibility to create device nodes for > > subdevs. > > > > But what you want is to basically pass whole structs as a control. That's > > something that ioctls where invented for. Especially once we have subdev > > nodes this shouldn't be a problem. > > > > Just the fact that it is easy to implement doesn't mean it should be done > > :-) > > > > Do you have specific use-cases for your proposed binary control? > > As discussed yesterday, here are a few use cases for the OMAP3 ISP driver. > > - white balance matrix > - gamma correction tables > > In both cases, the driver needs an array (possible 2 dimensional) of values to > configure the hardware. > > This can obviously be done using private ioctls, but what makes the red&blue > white balance gains different from the white balance matrix ? Why should the > first be controls and the later not ? I'll suggest some tests: 1. Can an operator easily manipulate the control in an intuitive fashion? 2. Can the kernel easily convey the visual form of the control to an application, such the application can render it with common widget sets not knowing much information apriori? So sliders for gain controls fit that criteria easily. I'm not sure about a matrix. I'm guessing one could have a "look up table" type control. It would visually (on a gui) be a matrix of text boxes that take numeric inputs; maybe with some kernel provided axes labels and row and column labaels. (I think that's better than an arbitrary binary blob, although more work to implement.) Regards, Andy ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 6:41 ` [PATCH/RFC 0/1] " Hans Verkuil 2010-03-30 8:57 ` Laurent Pinchart @ 2010-03-30 12:26 ` Kamil Debski 2010-03-30 13:14 ` Mauro Carvalho Chehab 2010-03-30 13:17 ` Laurent Pinchart 1 sibling, 2 replies; 10+ messages in thread From: Kamil Debski @ 2010-03-30 12:26 UTC (permalink / raw) To: 'Hans Verkuil'; +Cc: linux-media, Pawel Osciak, kyungmin.park > From: Hans Verkuil [mailto:hverkuil@xs4all.nl] > > Hi Kamil! Hi Hans, > > On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > > > > Hello, > > > > This patch introduces new type of v4l2 control - the binary control. > It > > will be useful for exchanging raw binary data between the user space > and > > the driver/hardware. > > > > The patch is pretty small – basically it adds a new control type. > > > > 1. Reasons to include this new type > > - Some devices require data which are not part of the stream, but > there > > are necessary for the device to work e.g. coefficients for > transformation > > matrices. > > - String control is not suitable as it suggests that the data is a > null > > terminated string. This might be important when printing debug > information - > > one might output strings as they are and binary data in hex. > > > > 2. How does the binary control work > > The binary control has been based on the string control. The > principle of > > use is the same. It uses v4l2_ext_control structure to pass the > pointer and > > size of the data. It is left for the driver to call the > copy_from_user/ > > copy_to_user function to copy the data. > > > > 3. About the patch > > The patch is pretty small – it basically adds a new control type. > > > > Best wishes, > > > > I don't think this is a good idea. Controls are not really meant to be > used > as an ioctl replacement. > > Controls can be used to control the hardware via a GUI (e.g. qv4l2). > Obviously, > this will fail for binary controls. Controls can also be used in cases > where > it is not known up front which controls are needed. This typically > happens for > bridge drivers that can use numerous combinations of i2c sub-devices. > Each > subdev can have its own controls. > > There is a grey area where you want to give the application access to > low-level > parameters but without showing them to the end-user. This is currently > not > possible, but it will be once the control framework is finished and > once we > have the possibility to create device nodes for subdevs. > > But what you want is to basically pass whole structs as a control. > That's > something that ioctls where invented for. Especially once we have > subdev nodes > this shouldn't be a problem. > > Just the fact that it is easy to implement doesn't mean it should be > done :-) > > Do you have specific use-cases for your proposed binary control? Yes, I have. I am working on a driver for a video codec which is using the mem2mem framework. I have to admit it's a pretty difficult hardware to work with. It was one of the reasons for Pawel Osciak to add multiplane support to videobuf. Before decoding, the hardware has to parse the header of the video stream to get all necessary parameters such as the number of buffers, width, height and some internal, codec specific stuff. The video stream is then demultiplexed and divided into encoded frames in software and the hardware can only process one, separated frame at a time. The whole codec setup cannot be achieved by using VIDIOC_S_FMT call, because hardware requires access to the header data. I wanted to use this binary control to pass the header to the codec after setting the right format with VIDIOC_S_FMT. Then video frames can be easily decoded as a standard calls to QBUF/DQBUF pairs. It is similar for encoding - the basic parameters are set with VIDIOC_S_FMT, then some codec specific/advanced are accessible as standard v4l2 controls. Then the encoding engine is initialized and the hardware returns an header of the output video stream. The header can be acquired by getting the value of the binary control. After that the frame to be encoded is provided and hw returns a single frame of encoded stream. Using custom ioctls seems appropriate for a hardware specific driver. Whereas the proposed binary control for getting and setting the video stream header could be generic solution and can be used in many drivers for hardware codecs. Do You have any better solution for such device? Best regards, -- Kamil Debski Linux Platform Group Samsung Poland R&D Center ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 12:26 ` Kamil Debski @ 2010-03-30 13:14 ` Mauro Carvalho Chehab 2010-03-30 14:32 ` Kamil Debski 2010-03-30 13:17 ` Laurent Pinchart 1 sibling, 1 reply; 10+ messages in thread From: Mauro Carvalho Chehab @ 2010-03-30 13:14 UTC (permalink / raw) To: Kamil Debski Cc: 'Hans Verkuil', linux-media, Pawel Osciak, kyungmin.park Kamil Debski wrote: >> From: Hans Verkuil [mailto:hverkuil@xs4all.nl] >> >> Hi Kamil! > > Hi Hans, > >> On Monday 29 March 2010 11:53:05 Kamil Debski wrote: >>> Hello, >>> >>> This patch introduces new type of v4l2 control - the binary control. >> It >>> will be useful for exchanging raw binary data between the user space >> and >>> the driver/hardware. >>> >>> The patch is pretty small – basically it adds a new control type. >>> >>> 1. Reasons to include this new type >>> - Some devices require data which are not part of the stream, but >> there >>> are necessary for the device to work e.g. coefficients for >> transformation >>> matrices. >>> - String control is not suitable as it suggests that the data is a >> null >>> terminated string. This might be important when printing debug >> information - >>> one might output strings as they are and binary data in hex. >>> >>> 2. How does the binary control work >>> The binary control has been based on the string control. The >> principle of >>> use is the same. It uses v4l2_ext_control structure to pass the >> pointer and >>> size of the data. It is left for the driver to call the >> copy_from_user/ >>> copy_to_user function to copy the data. >>> >>> 3. About the patch >>> The patch is pretty small – it basically adds a new control type. >>> >>> Best wishes, >>> >> I don't think this is a good idea. Controls are not really meant to be >> used >> as an ioctl replacement. >> >> Controls can be used to control the hardware via a GUI (e.g. qv4l2). >> Obviously, >> this will fail for binary controls. Controls can also be used in cases >> where >> it is not known up front which controls are needed. This typically >> happens for >> bridge drivers that can use numerous combinations of i2c sub-devices. >> Each >> subdev can have its own controls. >> >> There is a grey area where you want to give the application access to >> low-level >> parameters but without showing them to the end-user. This is currently >> not >> possible, but it will be once the control framework is finished and >> once we >> have the possibility to create device nodes for subdevs. >> >> But what you want is to basically pass whole structs as a control. >> That's >> something that ioctls where invented for. Especially once we have >> subdev nodes >> this shouldn't be a problem. >> >> Just the fact that it is easy to implement doesn't mean it should be >> done :-) >> >> Do you have specific use-cases for your proposed binary control? > > Yes, I have. I am working on a driver for a video codec which is using > the mem2mem framework. I have to admit it's a pretty difficult > hardware to work with. It was one of the reasons for Pawel Osciak > to add multiplane support to videobuf. > > Before decoding, the hardware has to parse the header of the video > stream to get all necessary parameters such as the number of buffers, > width, height and some internal, codec specific stuff. The video stream > is then demultiplexed and divided into encoded frames in software and > the hardware can only process one, separated frame at a time. > > The whole codec setup cannot be achieved by using VIDIOC_S_FMT call, > because hardware requires access to the header data. I wanted to use > this binary control to pass the header to the codec after setting the > right format with VIDIOC_S_FMT. Then video frames can be easily decoded > as a standard calls to QBUF/DQBUF pairs. > > It is similar for encoding - the basic parameters are set with > VIDIOC_S_FMT, then some codec specific/advanced are accessible as > standard v4l2 controls. Then the encoding engine is initialized and the > hardware returns an header of the output video stream. The header can > be acquired by getting the value of the binary control. After that the > frame to be encoded is provided and hw returns a single frame of > encoded stream. > > Using custom ioctls seems appropriate for a hardware specific driver. > Whereas the proposed binary control for getting and setting the video > stream header could be generic solution and can be used in many drivers > for hardware codecs. > > Do You have any better solution for such device? For sure using custom CTRL's is wrong. If I understood correctly, you want to send some mpeg transport stream (or something like) to the hardware and let it decode, right? For sure the current API spec don't cover such usecase. IMO, whatever decided, we need to add an example for this non-trivial usage. On a first glance, by using what we currently have for V4L2 API, it seems that the more correct approach is to format with S_FMT as MPEG, using a "resolution" that will allocate a buffer with enough size for sending the stream to the hardware (this is basically the way other mpeg-capable encoders/decoders do). Then, send the header via the usual stream interface and retrieve the stream information via VIDIOC_G_* ioctls. Only after having the header processed by the hardware, you'll be able to mmap memory to receive the decoded streams. I don't think we should add a custom ioctl for this case, as other hardware may have similar requirements, but maybe we should, instead, just create a new set of ioctl's for video processing. -- Cheers, Mauro ^ permalink raw reply [flat|nested] 10+ messages in thread
* RE: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 13:14 ` Mauro Carvalho Chehab @ 2010-03-30 14:32 ` Kamil Debski 2010-03-30 15:15 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 10+ messages in thread From: Kamil Debski @ 2010-03-30 14:32 UTC (permalink / raw) To: 'Mauro Carvalho Chehab' Cc: 'Hans Verkuil', linux-media, Pawel Osciak, kyungmin.park Hi Mauro, Mauro Carvalho Chehab wrote: > Kamil Debski wrote: > >> From: Hans Verkuil [mailto:hverkuil@xs4all.nl] > >> > >> Hi Kamil! > > > > Hi Hans, > > > >> On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > >>> Hello, > >>> > >>> This patch introduces new type of v4l2 control - the binary > control. > >> It > >>> will be useful for exchanging raw binary data between the user > space > >> and > >>> the driver/hardware. > >>> > >>> The patch is pretty small – basically it adds a new control type. > >>> > >>> 1. Reasons to include this new type > >>> - Some devices require data which are not part of the stream, but > >> there > >>> are necessary for the device to work e.g. coefficients for > >> transformation > >>> matrices. > >>> - String control is not suitable as it suggests that the data is a > >> null > >>> terminated string. This might be important when printing debug > >> information - > >>> one might output strings as they are and binary data in hex. > >>> > >>> 2. How does the binary control work > >>> The binary control has been based on the string control. The > >> principle of > >>> use is the same. It uses v4l2_ext_control structure to pass the > >> pointer and > >>> size of the data. It is left for the driver to call the > >> copy_from_user/ > >>> copy_to_user function to copy the data. > >>> > >>> 3. About the patch > >>> The patch is pretty small – it basically adds a new control type. > >>> > >>> Best wishes, > >>> > >> I don't think this is a good idea. Controls are not really meant to > be > >> used > >> as an ioctl replacement. > >> > >> Controls can be used to control the hardware via a GUI (e.g. qv4l2). > >> Obviously, > >> this will fail for binary controls. Controls can also be used in > cases > >> where > >> it is not known up front which controls are needed. This typically > >> happens for > >> bridge drivers that can use numerous combinations of i2c sub- > devices. > >> Each > >> subdev can have its own controls. > >> > >> There is a grey area where you want to give the application access > to > >> low-level > >> parameters but without showing them to the end-user. This is > currently > >> not > >> possible, but it will be once the control framework is finished and > >> once we > >> have the possibility to create device nodes for subdevs. > >> > >> But what you want is to basically pass whole structs as a control. > >> That's > >> something that ioctls where invented for. Especially once we have > >> subdev nodes > >> this shouldn't be a problem. > >> > >> Just the fact that it is easy to implement doesn't mean it should be > >> done :-) > >> > >> Do you have specific use-cases for your proposed binary control? > > > > Yes, I have. I am working on a driver for a video codec which is > using > > the mem2mem framework. I have to admit it's a pretty difficult > > hardware to work with. It was one of the reasons for Pawel Osciak > > to add multiplane support to videobuf. > > > > Before decoding, the hardware has to parse the header of the video > > stream to get all necessary parameters such as the number of buffers, > > width, height and some internal, codec specific stuff. The video > stream > > is then demultiplexed and divided into encoded frames in software and > > the hardware can only process one, separated frame at a time. > > > > The whole codec setup cannot be achieved by using VIDIOC_S_FMT call, > > because hardware requires access to the header data. I wanted to use > > this binary control to pass the header to the codec after setting the > > right format with VIDIOC_S_FMT. Then video frames can be easily > decoded > > as a standard calls to QBUF/DQBUF pairs. > > > > It is similar for encoding - the basic parameters are set with > > VIDIOC_S_FMT, then some codec specific/advanced are accessible as > > standard v4l2 controls. Then the encoding engine is initialized and > the > > hardware returns an header of the output video stream. The header can > > be acquired by getting the value of the binary control. After that > the > > frame to be encoded is provided and hw returns a single frame of > > encoded stream. > > > > Using custom ioctls seems appropriate for a hardware specific driver. > > Whereas the proposed binary control for getting and setting the video > > stream header could be generic solution and can be used in many > drivers > > for hardware codecs. > > > > Do You have any better solution for such device? > > For sure using custom CTRL's is wrong. > > If I understood correctly, you want to send some mpeg transport stream > (or something like) to the hardware and let it decode, right? > > For sure the current API spec don't cover such usecase. IMO, whatever > decided, > we need to add an example for this non-trivial usage. > > On a first glance, by using what we currently have for V4L2 API, it > seems that > the more correct approach is to format with S_FMT as MPEG, using a > "resolution" > that will allocate a buffer with enough size for sending the stream > to the hardware (this is basically the way other mpeg-capable > encoders/decoders > do). Then, send the header via the usual stream interface and retrieve > the stream information via VIDIOC_G_* ioctls. > > Only after having the header processed by the hardware, you'll be able > to mmap > memory to receive the decoded streams. > > I don't think we should add a custom ioctl for this case, as other > hardware may > have similar requirements, but maybe we should, instead, just create a > new set of > ioctl's for video processing. We are using elementary streams which are divided in software to header and encoded frames. The idea behind the binary control was keeping consistence between 1 QBUF on source video stream and 1 QBUF on output video stream. That was my motivation for the binary control. When decoding, the header would be passed to the driver with the binary control, and when encoding it would be read through the binary control. If I understand your suggestion, decoding at the beginning would require an extra QBUF/DQBUF on source video stream, before mmaping the buffers for output video frames. Similar for encoding - one extra QBUF/DQBUF for output stream at the beginning. It seems reasonable, if I discard the assumption that a QBUF on one queue (source) corresponds to one QBUF on the other queue (output). Best regards, -- Kamil Debski Linux Platform Group Samsung Poland R&D Center ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 14:32 ` Kamil Debski @ 2010-03-30 15:15 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 10+ messages in thread From: Mauro Carvalho Chehab @ 2010-03-30 15:15 UTC (permalink / raw) To: Kamil Debski Cc: 'Hans Verkuil', linux-media, Pawel Osciak, kyungmin.park Kamil Debski wrote: > Hi Mauro, > > Mauro Carvalho Chehab wrote: >> Kamil Debski wrote: >>>> From: Hans Verkuil [mailto:hverkuil@xs4all.nl] >>>> >>>> Hi Kamil! >>> Hi Hans, >>> >>>> On Monday 29 March 2010 11:53:05 Kamil Debski wrote: >>>>> Hello, >>>>> >>>>> This patch introduces new type of v4l2 control - the binary >> control. >>>> It >>>>> will be useful for exchanging raw binary data between the user >> space >>>> and >>>>> the driver/hardware. >>>>> >>>>> The patch is pretty small – basically it adds a new control type. >>>>> >>>>> 1. Reasons to include this new type >>>>> - Some devices require data which are not part of the stream, but >>>> there >>>>> are necessary for the device to work e.g. coefficients for >>>> transformation >>>>> matrices. >>>>> - String control is not suitable as it suggests that the data is a >>>> null >>>>> terminated string. This might be important when printing debug >>>> information - >>>>> one might output strings as they are and binary data in hex. >>>>> >>>>> 2. How does the binary control work >>>>> The binary control has been based on the string control. The >>>> principle of >>>>> use is the same. It uses v4l2_ext_control structure to pass the >>>> pointer and >>>>> size of the data. It is left for the driver to call the >>>> copy_from_user/ >>>>> copy_to_user function to copy the data. >>>>> >>>>> 3. About the patch >>>>> The patch is pretty small – it basically adds a new control type. >>>>> >>>>> Best wishes, >>>>> >>>> I don't think this is a good idea. Controls are not really meant to >> be >>>> used >>>> as an ioctl replacement. >>>> >>>> Controls can be used to control the hardware via a GUI (e.g. qv4l2). >>>> Obviously, >>>> this will fail for binary controls. Controls can also be used in >> cases >>>> where >>>> it is not known up front which controls are needed. This typically >>>> happens for >>>> bridge drivers that can use numerous combinations of i2c sub- >> devices. >>>> Each >>>> subdev can have its own controls. >>>> >>>> There is a grey area where you want to give the application access >> to >>>> low-level >>>> parameters but without showing them to the end-user. This is >> currently >>>> not >>>> possible, but it will be once the control framework is finished and >>>> once we >>>> have the possibility to create device nodes for subdevs. >>>> >>>> But what you want is to basically pass whole structs as a control. >>>> That's >>>> something that ioctls where invented for. Especially once we have >>>> subdev nodes >>>> this shouldn't be a problem. >>>> >>>> Just the fact that it is easy to implement doesn't mean it should be >>>> done :-) >>>> >>>> Do you have specific use-cases for your proposed binary control? >>> Yes, I have. I am working on a driver for a video codec which is >> using >>> the mem2mem framework. I have to admit it's a pretty difficult >>> hardware to work with. It was one of the reasons for Pawel Osciak >>> to add multiplane support to videobuf. >>> >>> Before decoding, the hardware has to parse the header of the video >>> stream to get all necessary parameters such as the number of buffers, >>> width, height and some internal, codec specific stuff. The video >> stream >>> is then demultiplexed and divided into encoded frames in software and >>> the hardware can only process one, separated frame at a time. >>> >>> The whole codec setup cannot be achieved by using VIDIOC_S_FMT call, >>> because hardware requires access to the header data. I wanted to use >>> this binary control to pass the header to the codec after setting the >>> right format with VIDIOC_S_FMT. Then video frames can be easily >> decoded >>> as a standard calls to QBUF/DQBUF pairs. >>> >>> It is similar for encoding - the basic parameters are set with >>> VIDIOC_S_FMT, then some codec specific/advanced are accessible as >>> standard v4l2 controls. Then the encoding engine is initialized and >> the >>> hardware returns an header of the output video stream. The header can >>> be acquired by getting the value of the binary control. After that >> the >>> frame to be encoded is provided and hw returns a single frame of >>> encoded stream. >>> >>> Using custom ioctls seems appropriate for a hardware specific driver. >>> Whereas the proposed binary control for getting and setting the video >>> stream header could be generic solution and can be used in many >> drivers >>> for hardware codecs. >>> >>> Do You have any better solution for such device? >> For sure using custom CTRL's is wrong. >> >> If I understood correctly, you want to send some mpeg transport stream >> (or something like) to the hardware and let it decode, right? >> >> For sure the current API spec don't cover such usecase. IMO, whatever >> decided, >> we need to add an example for this non-trivial usage. >> >> On a first glance, by using what we currently have for V4L2 API, it >> seems that >> the more correct approach is to format with S_FMT as MPEG, using a >> "resolution" >> that will allocate a buffer with enough size for sending the stream >> to the hardware (this is basically the way other mpeg-capable >> encoders/decoders >> do). Then, send the header via the usual stream interface and retrieve >> the stream information via VIDIOC_G_* ioctls. >> >> Only after having the header processed by the hardware, you'll be able >> to mmap >> memory to receive the decoded streams. >> >> I don't think we should add a custom ioctl for this case, as other >> hardware may >> have similar requirements, but maybe we should, instead, just create a >> new set of >> ioctl's for video processing. > > We are using elementary streams which are divided in software to header > and encoded frames. The idea behind the binary control was keeping > consistence between 1 QBUF on source video stream and 1 QBUF on output > video stream. That was my motivation for the binary control. > > When decoding, the header would be passed to the driver with the binary > control, and when encoding it would be read through the binary control. > > If I understand your suggestion, decoding at the beginning would require > an extra QBUF/DQBUF on source video stream, before mmaping the buffers for > output video frames. Similar for encoding - one extra QBUF/DQBUF for output > stream at the beginning. Yes, that's what I'm proposing as a way for the decoder to automatically detect the format of the input stream. For encoding, the better approach is to use the CTRLs that control the MPEG parameters, adding more if your hardware support other formats than what's currently available at the API. In this case, create one control per variable that needs to be configured (bit stream, audio format, video format, etc). In any case, reading the encoder/decoder parameters via CTRL should return the format used by the hardware. > It seems reasonable, if I discard the assumption that a QBUF on one queue > (source) corresponds to one QBUF on the other queue (output). I don't think you should have any type of assumption between the input and the output. I can imagine some cases where the number of buffers will be different. For example, if you have a TS with multiple PID's inside, some mem2mem hardware may be able to filter the PID's. So, at the output, you'll have less buffers. Cheers, Mauro ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH/RFC 0/1] v4l: Add support for binary controls 2010-03-30 12:26 ` Kamil Debski 2010-03-30 13:14 ` Mauro Carvalho Chehab @ 2010-03-30 13:17 ` Laurent Pinchart 1 sibling, 0 replies; 10+ messages in thread From: Laurent Pinchart @ 2010-03-30 13:17 UTC (permalink / raw) To: Kamil Debski Cc: 'Hans Verkuil', linux-media, Pawel Osciak, kyungmin.park Hi Kamil, On Tuesday 30 March 2010 14:26:00 Kamil Debski wrote: > > From: Hans Verkuil [mailto:hverkuil@xs4all.nl] > > On Monday 29 March 2010 11:53:05 Kamil Debski wrote: > > > Hello, > > > > > > This patch introduces new type of v4l2 control - the binary control. It > > > will be useful for exchanging raw binary data between the user space and > > > the driver/hardware. > > > > > > The patch is pretty small – basically it adds a new control type. > > > > > > 1. Reasons to include this new type > > > - Some devices require data which are not part of the stream, but there > > > are necessary for the device to work e.g. coefficients for > > > transformation matrices. > > > - String control is not suitable as it suggests that the data is a > > > null terminated string. This might be important when printing debug > > > information - one might output strings as they are and binary data in > > > hex. > > > > > > 2. How does the binary control work > > > The binary control has been based on the string control. The principle > > > of use is the same. It uses v4l2_ext_control structure to pass the > > > pointer and size of the data. It is left for the driver to call the > > > copy_from_user/copy_to_user function to copy the data. > > > > > > 3. About the patch > > > The patch is pretty small – it basically adds a new control type. > > > > > > > I don't think this is a good idea. Controls are not really meant to be > > used as an ioctl replacement. > > > > Controls can be used to control the hardware via a GUI (e.g. qv4l2). > > Obviously, this will fail for binary controls. Controls can also be used > > in cases where it is not known up front which controls are needed. This > > typically happens for bridge drivers that can use numerous combinations of > > i2c sub-devices. Each subdev can have its own controls. > > > > There is a grey area where you want to give the application access to > > low-level parameters but without showing them to the end-user. This is > > currently not possible, but it will be once the control framework is > > finished and once we have the possibility to create device nodes for > > subdevs. > > > > But what you want is to basically pass whole structs as a control. > > That's something that ioctls where invented for. Especially once we have > > subdev nodes this shouldn't be a problem. > > > > Just the fact that it is easy to implement doesn't mean it should be > > done :-) > > > > Do you have specific use-cases for your proposed binary control? > > Yes, I have. I am working on a driver for a video codec which is using > the mem2mem framework. I have to admit it's a pretty difficult > hardware to work with. It was one of the reasons for Pawel Osciak > to add multiplane support to videobuf. > > Before decoding, the hardware has to parse the header of the video > stream to get all necessary parameters such as the number of buffers, > width, height and some internal, codec specific stuff. The video stream > is then demultiplexed and divided into encoded frames in software and > the hardware can only process one, separated frame at a time. > > The whole codec setup cannot be achieved by using VIDIOC_S_FMT call, > because hardware requires access to the header data. I wanted to use > this binary control to pass the header to the codec after setting the > right format with VIDIOC_S_FMT. Then video frames can be easily decoded > as a standard calls to QBUF/DQBUF pairs. In that case I have to agree with Hans, a private ioctl is better. > It is similar for encoding - the basic parameters are set with > VIDIOC_S_FMT, then some codec specific/advanced are accessible as > standard v4l2 controls. Then the encoding engine is initialized and the > hardware returns an header of the output video stream. The header can > be acquired by getting the value of the binary control. After that the > frame to be encoded is provided and hw returns a single frame of > encoded stream. > > Using custom ioctls seems appropriate for a hardware specific driver. > Whereas the proposed binary control for getting and setting the video > stream header could be generic solution and can be used in many drivers > for hardware codecs. I don't think it would. The format of the data is pretty much hardware specific. If you can make the format generic, then the private ioctl could become generic. > Do You have any better solution for such device? -- Regards, Laurent Pinchart ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2010-03-30 15:16 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-03-29 9:53 [PATCH/RFC 0/1] v4l: Add support for binary controls Kamil Debski 2010-03-29 9:53 ` [PATCH/RFC 1/1] " Kamil Debski 2010-03-30 6:41 ` [PATCH/RFC 0/1] " Hans Verkuil 2010-03-30 8:57 ` Laurent Pinchart 2010-03-30 11:43 ` Andy Walls 2010-03-30 12:26 ` Kamil Debski 2010-03-30 13:14 ` Mauro Carvalho Chehab 2010-03-30 14:32 ` Kamil Debski 2010-03-30 15:15 ` Mauro Carvalho Chehab 2010-03-30 13:17 ` Laurent Pinchart
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.