[PATCH v3 0/2] v4l: Add camera voice coil lens control class, current control

[PATCH v3 0/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

Hello everyone,

I wanted to refresh my voice coil lens patchset before we have more voice 
coil lens controller drivers. The VOCUS_ABSOLUTE control really is not a
best control ID to control a voice coil driver's current. 

There may be additional controls in the class: the hardware I'm familiar
with provides other controls (PWM vs. linear mode, resonance frequency and
ringing compensation formula to name a few) but I'm not fully certain 
they're something that even should be told to the user --- let alone
giving the user write access to them. 

My expectation is still that there will be more controls in the class. The
PWM / linear mode might be one candidate: PWM saves power but it may cause
other issues. These other issues might be something to ignore, depending
on the use case. That will be anyway left for the future. 

since v2:

- Don't remove the newline after control class definitions.

- Move all ad5820 related changes to the second patch. Some were
  accidentally left to the patch adding the new control class.

-- 
Kind regards,
Sakari

[PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

Add a V4L2 control class for voice coil lens driver devices. These are
simple devices that are used to move a camera lens from its resting
position.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 Documentation/media/uapi/v4l/extended-controls.rst | 28 ++++++++++++++++++++++
 include/uapi/linux/v4l2-controls.h                 |  8 +++++++
 2 files changed, 36 insertions(+)

diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst
index abb1057..a75451a 100644
--- a/Documentation/media/uapi/v4l/extended-controls.rst
+++ b/Documentation/media/uapi/v4l/extended-controls.rst
@@ -3022,6 +3022,34 @@ Image Process Control IDs
     driver specific and are documented in :ref:`v4l-drivers`.
 
 
+.. _voice-coil-lens-controls:
+
+Voice Coil Lens Control Reference
+=================================
+
+The Voice Coil class controls are used to control voice coil lens
+devices. These are very simple devices that consist of a voice coil, a
+spring and a lens. The current applied on the voice coil is used to
+move the lens away from the resting position which typically is (close
+to) infinity. The higher the current applied, the closer the lens is
+typically focused.
+
+.. _voice-coil-lens-control-is:
+
+Voice Coil Lens Control IDs
+---------------------------
+
+``V4L2_CID_VOICE_COIL_CLASS (class)``
+    The VOICE_COIL class descriptor.
+
+``V4L2_CID_VOICE_COIL_CURRENT (integer)``
+    Current applied on a voice coil. The more current is applied, the
+    more is the position of the lens moved from its resting position.
+    Do note that there may be a ringing effect; the lens will
+    oscillate after changing the current applied unless the device
+    implements ringing compensation.
+
+
 .. _dv-controls:
 
 Digital Video Control Reference
diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
index 0d2e1e0..9ef152b 100644
--- a/include/uapi/linux/v4l2-controls.h
+++ b/include/uapi/linux/v4l2-controls.h
@@ -62,6 +62,7 @@
 #define V4L2_CTRL_CLASS_FM_RX		0x00a10000	/* FM Receiver controls */
 #define V4L2_CTRL_CLASS_RF_TUNER	0x00a20000	/* RF tuner controls */
 #define V4L2_CTRL_CLASS_DETECT		0x00a30000	/* Detection controls */
+#define V4L2_CTRL_CLASS_VOICE_COIL	0x00a40000	/* Voice coil lens driver controls */
 
 /* User-class control IDs */
 
@@ -894,6 +895,13 @@ enum v4l2_jpeg_chroma_subsampling {
 #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
 #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
 
+/* Voice coil lens driver controls */
+
+#define V4L2_CID_VOICE_COIL_CLASS_BASE		(V4L2_CTRL_CLASS_VOICE_COIL | 0x900)
+#define V4L2_CID_VOICE_COIL_CLASS		(V4L2_CTRL_CLASS_VOICE_COIL | 1)
+
+#define V4L2_CID_VOICE_COIL_CURRENT		(V4L2_CID_VOICE_COIL_CLASS_BASE + 1)
+
 
 /*  DV-class control IDs defined by V4L2 */
 #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)
-- 
2.7.4

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3563 bytes --]

On Tue 2017-02-14 14:20:22, Sakari Ailus wrote:
> Add a V4L2 control class for voice coil lens driver devices. These are
> simple devices that are used to move a camera lens from its resting
> position.
> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>

Looks good to me.

I wonder... should we somehow expose the range of diopters to
userspace? I believe userland camera application will need that
information.

Thanks,
							Pavel

> ---
>  Documentation/media/uapi/v4l/extended-controls.rst | 28 ++++++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h                 |  8 +++++++
>  2 files changed, 36 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst
> index abb1057..a75451a 100644
> --- a/Documentation/media/uapi/v4l/extended-controls.rst
> +++ b/Documentation/media/uapi/v4l/extended-controls.rst
> @@ -3022,6 +3022,34 @@ Image Process Control IDs
>      driver specific and are documented in :ref:`v4l-drivers`.
>  
>  
> +.. _voice-coil-lens-controls:
> +
> +Voice Coil Lens Control Reference
> +=================================
> +
> +The Voice Coil class controls are used to control voice coil lens
> +devices. These are very simple devices that consist of a voice coil, a
> +spring and a lens. The current applied on the voice coil is used to
> +move the lens away from the resting position which typically is (close
> +to) infinity. The higher the current applied, the closer the lens is
> +typically focused.
> +
> +.. _voice-coil-lens-control-is:
> +
> +Voice Coil Lens Control IDs
> +---------------------------
> +
> +``V4L2_CID_VOICE_COIL_CLASS (class)``
> +    The VOICE_COIL class descriptor.
> +
> +``V4L2_CID_VOICE_COIL_CURRENT (integer)``
> +    Current applied on a voice coil. The more current is applied, the
> +    more is the position of the lens moved from its resting position.
> +    Do note that there may be a ringing effect; the lens will
> +    oscillate after changing the current applied unless the device
> +    implements ringing compensation.
> +
> +
>  .. _dv-controls:
>  
>  Digital Video Control Reference
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 0d2e1e0..9ef152b 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -62,6 +62,7 @@
>  #define V4L2_CTRL_CLASS_FM_RX		0x00a10000	/* FM Receiver controls */
>  #define V4L2_CTRL_CLASS_RF_TUNER	0x00a20000	/* RF tuner controls */
>  #define V4L2_CTRL_CLASS_DETECT		0x00a30000	/* Detection controls */
> +#define V4L2_CTRL_CLASS_VOICE_COIL	0x00a40000	/* Voice coil lens driver controls */
>  
>  /* User-class control IDs */
>  
> @@ -894,6 +895,13 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
>  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
>  
> +/* Voice coil lens driver controls */
> +
> +#define V4L2_CID_VOICE_COIL_CLASS_BASE		(V4L2_CTRL_CLASS_VOICE_COIL | 0x900)
> +#define V4L2_CID_VOICE_COIL_CLASS		(V4L2_CTRL_CLASS_VOICE_COIL | 1)
> +
> +#define V4L2_CID_VOICE_COIL_CURRENT		(V4L2_CID_VOICE_COIL_CLASS_BASE + 1)
> +
>  
>  /*  DV-class control IDs defined by V4L2 */
>  #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

[-- Attachment #1.1: Type: text/plain, Size: 903 bytes --]

Hi Pavel,

On 02/15/17 00:47, Pavel Machek wrote:
> On Tue 2017-02-14 14:20:22, Sakari Ailus wrote:
>> Add a V4L2 control class for voice coil lens driver devices. These are
>> simple devices that are used to move a camera lens from its resting
>> position.
>>
>> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> 
> Looks good to me.
> 
> I wonder... should we somehow expose the range of diopters to
> userspace? I believe userland camera application will need that
> information.

It'd certainly be useful to be able to provide more information.

The question is: where to store it, and how? It depends on the voice
coil, the spring constant, the lens and the distance of the lens from
the sensor --- at least. Probably the sensor size as well.

On voice coil lenses it is also somewhat inexact.

-- 
Kind regards,

Sakari Ailus
sakari.ailus@linux.intel.com


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 213 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1316 bytes --]

Hi!

> On 02/15/17 00:47, Pavel Machek wrote:
> > On Tue 2017-02-14 14:20:22, Sakari Ailus wrote:
> >> Add a V4L2 control class for voice coil lens driver devices. These are
> >> simple devices that are used to move a camera lens from its resting
> >> position.
> >>
> >> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > 
> > Looks good to me.
> > 
> > I wonder... should we somehow expose the range of diopters to
> > userspace? I believe userland camera application will need that
> > information.
> 
> It'd certainly be useful to be able to provide more information.
> 
> The question is: where to store it, and how? It depends on the voice
> coil, the spring constant, the lens and the distance of the lens from
> the sensor --- at least. Probably the sensor size as well.
> 
> On voice coil lenses it is also somewhat inexact.

I was thinking read-only attribute providing minimum and maximum
diopters in case there's linear relationship as on N900.

+#define V4L2_CID_VOICE_DIOPTERS_AT_REST (V4L2_CID_VOICE_COIL_CLASS_BASE + 2)
+#define V4L2_CID_VOICE_DIOPTERS_AT_MAX (V4L2_CID_VOICE_COIL_CLASS_BASE + 3)

?

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

Hi Pavel,

On Wed, Feb 15, 2017 at 09:09:09AM +0100, Pavel Machek wrote:
> Hi!
> 
> > On 02/15/17 00:47, Pavel Machek wrote:
> > > On Tue 2017-02-14 14:20:22, Sakari Ailus wrote:
> > >> Add a V4L2 control class for voice coil lens driver devices. These are
> > >> simple devices that are used to move a camera lens from its resting
> > >> position.
> > >>
> > >> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> > > 
> > > Looks good to me.
> > > 
> > > I wonder... should we somehow expose the range of diopters to
> > > userspace? I believe userland camera application will need that
> > > information.
> > 
> > It'd certainly be useful to be able to provide more information.
> > 
> > The question is: where to store it, and how? It depends on the voice
> > coil, the spring constant, the lens and the distance of the lens from
> > the sensor --- at least. Probably the sensor size as well.
> > 
> > On voice coil lenses it is also somewhat inexact.
> 
> I was thinking read-only attribute providing minimum and maximum
> diopters in case there's linear relationship as on N900.
> 
> +#define V4L2_CID_VOICE_DIOPTERS_AT_REST (V4L2_CID_VOICE_COIL_CLASS_BASE + 2)
> +#define V4L2_CID_VOICE_DIOPTERS_AT_MAX (V4L2_CID_VOICE_COIL_CLASS_BASE + 3)

Where do you store that information and how? Should the user be also told
how the applied current affects the value?

I also wonder whether that's the best way to provide the information to the
user --- we have things such as devices that are a part of a camera module
and telling the user on which side of the device the camera is located.

We've been planning to have a property API for this to provide the user with
a tree of key-value pairs, with details unsettled as of yet, so it's
certainly nothing that could be used yet.

Do you have a user application that could make use of such information?

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2498 bytes --]

Hi!

> > > > I wonder... should we somehow expose the range of diopters to
> > > > userspace? I believe userland camera application will need that
> > > > information.
> > > 
> > > It'd certainly be useful to be able to provide more information.
> > > 
> > > The question is: where to store it, and how? It depends on the voice
> > > coil, the spring constant, the lens and the distance of the lens from
> > > the sensor --- at least. Probably the sensor size as well.
> > > 
> > > On voice coil lenses it is also somewhat inexact.
> > 
> > I was thinking read-only attribute providing minimum and maximum
> > diopters in case there's linear relationship as on N900.
> > 
> > +#define V4L2_CID_VOICE_DIOPTERS_AT_REST (V4L2_CID_VOICE_COIL_CLASS_BASE + 2)
> > +#define V4L2_CID_VOICE_DIOPTERS_AT_MAX (V4L2_CID_VOICE_COIL_CLASS_BASE + 3)
> 
> Where do you store that information and how? Should the user be also told
> how the applied current affects the value?

The information would come from device tree.

User already knows minimum and maximum, so if he knows there's linear
relationship, he has complete picture. I'm not sure if there are some
voice coils with anything else then linear relationship. I guess we
could do

+#define V4L2_CID_VOICE_CURRENT_TO_DIOPTERS (V4L2_CID_VOICE_COIL_CLASS_BASE + 4)
#define ..._LINEAR 1

...

> I also wonder whether that's the best way to provide the information to the
> user --- we have things such as devices that are a part of a camera module
> and telling the user on which side of the device the camera is located.
> 
> We've been planning to have a property API for this to provide the user with
> a tree of key-value pairs, with details unsettled as of yet, so it's
> certainly nothing that could be used yet.

You know the design better than I do. I believe read-only properties
would be easy enough for this.

> Do you have a user application that could make use of such information?

fcam-dev,
yes. https://gitlab.com/pavelm/fcam-devhttps://gitlab.com/pavelm/fcam-dev

I don't promise to make neccessary modifications -- currently it
hardcodes 0 and 20 diopters -- but it allows manual or automatic
focus; in both modes it is good to tell user where he is focusing, and
it is pretty much mandatory in the manual mode.

Best regards,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Mauro Carvalho Chehab]

Hi Sakari,

Em Tue, 14 Feb 2017 14:20:22 +0200
Sakari Ailus <sakari.ailus@linux.intel.com> escreveu:

> Add a V4L2 control class for voice coil lens driver devices. These are
> simple devices that are used to move a camera lens from its resting
> position.

>From some past threads with this patch, you mentioned that:

"The FOCUS_ABSOLUTE control really is not a best control ID to
 control a voice coil driver's current."

However, I'm not seeing any explanation there at the thread, at
the patch description or at the documentation explaining why, and,
more important, when someone should use the focus control or the
camera voice coil control.

Worse than that, patch 2/2 gives the false sensation that both
controls are equal.

Ok, I understand that they need to be identical on the existing
driver, in order to keep backward compatibility, but I'm afraid
that, without a clear distinction between them at the documentation,
people may just clone the existing code on other drivers.

So, please add more details to patch 1/2.

Regards,
Mauro

> 
> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
> ---
>  Documentation/media/uapi/v4l/extended-controls.rst | 28 ++++++++++++++++++++++
>  include/uapi/linux/v4l2-controls.h                 |  8 +++++++
>  2 files changed, 36 insertions(+)
> 
> diff --git a/Documentation/media/uapi/v4l/extended-controls.rst b/Documentation/media/uapi/v4l/extended-controls.rst
> index abb1057..a75451a 100644
> --- a/Documentation/media/uapi/v4l/extended-controls.rst
> +++ b/Documentation/media/uapi/v4l/extended-controls.rst
> @@ -3022,6 +3022,34 @@ Image Process Control IDs
>      driver specific and are documented in :ref:`v4l-drivers`.
>  
>  
> +.. _voice-coil-lens-controls:
> +
> +Voice Coil Lens Control Reference
> +=================================
> +
> +The Voice Coil class controls are used to control voice coil lens
> +devices. These are very simple devices that consist of a voice coil, a
> +spring and a lens. The current applied on the voice coil is used to
> +move the lens away from the resting position which typically is (close
> +to) infinity. The higher the current applied, the closer the lens is
> +typically focused.
> +
> +.. _voice-coil-lens-control-is:
> +
> +Voice Coil Lens Control IDs
> +---------------------------
> +
> +``V4L2_CID_VOICE_COIL_CLASS (class)``
> +    The VOICE_COIL class descriptor.
> +
> +``V4L2_CID_VOICE_COIL_CURRENT (integer)``
> +    Current applied on a voice coil. The more current is applied, the
> +    more is the position of the lens moved from its resting position.
> +    Do note that there may be a ringing effect; the lens will
> +    oscillate after changing the current applied unless the device
> +    implements ringing compensation.
> +
> +
>  .. _dv-controls:
>  
>  Digital Video Control Reference
> diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h
> index 0d2e1e0..9ef152b 100644
> --- a/include/uapi/linux/v4l2-controls.h
> +++ b/include/uapi/linux/v4l2-controls.h
> @@ -62,6 +62,7 @@
>  #define V4L2_CTRL_CLASS_FM_RX		0x00a10000	/* FM Receiver controls */
>  #define V4L2_CTRL_CLASS_RF_TUNER	0x00a20000	/* RF tuner controls */
>  #define V4L2_CTRL_CLASS_DETECT		0x00a30000	/* Detection controls */
> +#define V4L2_CTRL_CLASS_VOICE_COIL	0x00a40000	/* Voice coil lens driver controls */
>  
>  /* User-class control IDs */
>  
> @@ -894,6 +895,13 @@ enum v4l2_jpeg_chroma_subsampling {
>  #define V4L2_CID_TEST_PATTERN			(V4L2_CID_IMAGE_PROC_CLASS_BASE + 3)
>  #define V4L2_CID_DEINTERLACING_MODE		(V4L2_CID_IMAGE_PROC_CLASS_BASE + 4)
>  
> +/* Voice coil lens driver controls */
> +
> +#define V4L2_CID_VOICE_COIL_CLASS_BASE		(V4L2_CTRL_CLASS_VOICE_COIL | 0x900)
> +#define V4L2_CID_VOICE_COIL_CLASS		(V4L2_CTRL_CLASS_VOICE_COIL | 1)
> +
> +#define V4L2_CID_VOICE_COIL_CURRENT		(V4L2_CID_VOICE_COIL_CLASS_BASE + 1)
> +
>  
>  /*  DV-class control IDs defined by V4L2 */
>  #define V4L2_CID_DV_CLASS_BASE			(V4L2_CTRL_CLASS_DV | 0x900)



Thanks,
Mauro

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 656 bytes --]

Hi!

> Worse than that, patch 2/2 gives the false sensation that both
> controls are equal.
> 
> Ok, I understand that they need to be identical on the existing
> driver, in order to keep backward compatibility, but I'm afraid
> that, without a clear distinction between them at the documentation,
> people may just clone the existing code on other drivers.

Actually, you don't need to workky about the backwards compatibility
there. I'm pretty sure I'm the only user, and I can adapt.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

Hi Mauro,

On Fri, Apr 14, 2017 at 11:23:32PM -0300, Mauro Carvalho Chehab wrote:
> Hi Sakari,
> 
> Em Tue, 14 Feb 2017 14:20:22 +0200
> Sakari Ailus <sakari.ailus@linux.intel.com> escreveu:
> 
> > Add a V4L2 control class for voice coil lens driver devices. These are
> > simple devices that are used to move a camera lens from its resting
> > position.
> 
> From some past threads with this patch, you mentioned that:
> 
> "The FOCUS_ABSOLUTE control really is not a best control ID to
>  control a voice coil driver's current."
> 
> However, I'm not seeing any explanation there at the thread, at
> the patch description or at the documentation explaining why, and,
> more important, when someone should use the focus control or the
> camera voice coil control.

It should be available in the thread. Nevertheless, V4L2_CID_FOCUS_ABSOLUTE
is documented as follows (emphasis mine):

	This control sets the *focal point* of the camera to the specified
	position. The unit is undefined. Positive values set the focus
	closer to the camera, negative values towards infinity.

What you control in voice coil devices is current (in Ampères) and the
current only has a relatively loose relation to the focal point.
Additionally, increasing the current brings the focus closer, not farther.

I anticipate adding controls for ringing compensation in the future.
Virtually all other devices except this one do ringing compensation and
there's some control to be done for that.

How about adding such an explanation added to the commit message?

> 
> Worse than that, patch 2/2 gives the false sensation that both
> controls are equal.
> 
> Ok, I understand that they need to be identical on the existing
> driver, in order to keep backward compatibility, but I'm afraid
> that, without a clear distinction between them at the documentation,
> people may just clone the existing code on other drivers.

Indeed. The only reason that I'm not just replacing FOCUS_ABSOLUTE with the
new contorol is backwards compatibility. But as Pavel pointed out, he's
likely the sole user of this device that can only be found (as far as we
commonly are aware) in the Nokia N900.

I'm happy to just switch the control, and Pavel mentioned he's happy with
that. It would avoid copying the code in new drivers --- which I would most
certainly point out anyway.

> 
> So, please add more details to patch 1/2.

Let me know if you're happy with the above.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Mauro Carvalho Chehab]

Em Sun, 16 Apr 2017 12:12:10 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> Hi Mauro,
> 
> On Fri, Apr 14, 2017 at 11:23:32PM -0300, Mauro Carvalho Chehab wrote:
> > Hi Sakari,
> > 
> > Em Tue, 14 Feb 2017 14:20:22 +0200
> > Sakari Ailus <sakari.ailus@linux.intel.com> escreveu:
> >   
> > > Add a V4L2 control class for voice coil lens driver devices. These are
> > > simple devices that are used to move a camera lens from its resting
> > > position.  
> > 
> > From some past threads with this patch, you mentioned that:
> > 
> > "The FOCUS_ABSOLUTE control really is not a best control ID to
> >  control a voice coil driver's current."
> > 
> > However, I'm not seeing any explanation there at the thread, at
> > the patch description or at the documentation explaining why, and,
> > more important, when someone should use the focus control or the
> > camera voice coil control.  
> 
> It should be available in the thread.

The email thread is not registered at git logs nor at the API spec.

> Nevertheless, V4L2_CID_FOCUS_ABSOLUTE
> is documented as follows (emphasis mine):
> 
> 	This control sets the *focal point* of the camera to the specified
> 	position. The unit is undefined. Positive values set the focus
> 	closer to the camera, negative values towards infinity.
> 
> What you control in voice coil devices is current (in Ampères) and the
> current only has a relatively loose relation to the focal point.

The real problem I'm seeing here is that this control is already
used by voice coil motor (VCM). Several UVC-based Logitech cameras
come with VCM, like their QuickCam Pro-series webcams:

	https://secure.logitech.com/en-hk/articles/3231

The voice coil can be seen on this picture:
	https://photo.stackexchange.com/questions/48678/can-i-modify-a-logitech-c615-webcam-for-infinity-focus

> Additionally, increasing the current brings the focus closer, not farther.

That's just a hardware/software implementation detail. One could use
a voice coil to do just the reverse: without any current, it would
be getting a minimal focus distance; with max current it would
go to infinite, or may have some firmware inside the hardware that
would be inverting the signal.

I have here a C920 camera. This model has V4L2_CID_FOCUS_ABSOLUTE:

ioctl(3, VIDIOC_G_EXT_CTRLS, {ctrl_class=V4L2_CTRL_CLASS_CAMERA, count=1, controls=[{id=V4L2_CID_FOCUS_ABSOLUTE, size=0, value=0, value64=0}]}) = 0
write(1, "                 focus_absolute "..., 97                 focus_absolute (int)    : min=0 max=250 step=5 default=0 value=0 flags=inactive

On this model, V4L2_CID_FOCUS_ABSOLUTE == infinite.

On a quick check at uvc driver, it seems that the driver itself
doesn't invert the value before sending to the device. So, 
I guess that, either the camera firmware do something like:
	current = 250 - control_value
Or the VCM here is mounted, in hardware, to use less current
when focusing on a close object.

Looking on it on another side, this control was added on this changeset:

    commit f9bd5843658e18a7097fc7258c60fb840109eaa8
    Author: Brandon Philips <bphilips@suse.de>
    Date:   Tue Apr 22 14:42:02 2008 -0300

    V4L/DVB (7167): [v4l] Add camera class control definitions

Meant to be used on USB cameras. Until this changeset:

	commit bee3d51156113363e952674504833b4bc92cf15e
	Author: Pavel Machek <pavel@ucw.cz>
	Date:   Fri Aug 5 07:26:11 2016 -0300

	[media] ad5820: Add driver for auto-focus coil

The only driver that were using it were uvcvideo.

As far as I remember, Brandon was working together with a Logitech
developer, in order to add support for those Quickcam Pro cameras.
So, what this control actually sets is the VCM.

That's why I don't see any need to add another control to do the same
thing.

> I anticipate adding controls for ringing compensation in the future.
> Virtually all other devices except this one do ringing compensation and
> there's some control to be done for that.

Hmm... if the idea is to have a control that doesn't do ringing
compensation, then it should be clear at the control's descriptions
that:

- V4L2_CID_FOCUS_ABSOLUTE should be used if the VCM has ringing
  compensation;
- V4L2_CID_VOICE_COIL_CURRENT and V4L2_CID_VOICE_COIL_RING_COMPENSATION
  should be used otherwise.

Btw, if the rationale for this patch is to support devices without
ring compensation, so, both controls and their descriptions should
be added at the same time, together with a patchset that would be
using both.

> How about adding such an explanation added to the commit message?

It is not enough. Documentation should be clear that VCM devices
with ring compensation should use V4L2_CID_FOCUS_ABSOLUTE.

> 
> > 
> > Worse than that, patch 2/2 gives the false sensation that both
> > controls are equal.
> > 
> > Ok, I understand that they need to be identical on the existing
> > driver, in order to keep backward compatibility, but I'm afraid
> > that, without a clear distinction between them at the documentation,
> > people may just clone the existing code on other drivers.  
> 
> Indeed. The only reason that I'm not just replacing FOCUS_ABSOLUTE with the
> new contorol is backwards compatibility. But as Pavel pointed out, he's
> likely the sole user of this device that can only be found (as far as we
> commonly are aware) in the Nokia N900.
> 
> I'm happy to just switch the control, and Pavel mentioned he's happy with
> that. It would avoid copying the code in new drivers --- which I would most
> certainly point out anyway.
> 
> > 
> > So, please add more details to patch 1/2.  
> 
> Let me know if you're happy with the above.
> 



Thanks,
Mauro

support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 42076 bytes --]

Hi!

For focus to be useful, we need autofocus implmented
somewhere. Unfortunately, v4l framework does not seem to provide good
place where to put autofocus. I believe, long-term, we'll need some
kind of "video server" providing this kind of services.

Anyway, we probably don't want autofocus in kernel (even through some
cameras do it in hardware), and we probably don't want autofocus in
each and every user application.

So what remains is libv4l2. Now, this is in no way clean or complete,
and functionality provided by sdl.c and asciicam.c probably _should_
be in application, but... I'd like to get the code out there.

Oh and yes, I've canibalized decode_tm6000.c application instead of
introducing my own. Autotools scare me, sorry.

Regards,
							Pavel

diff --git a/lib/libv4l2/asciicam.c b/lib/libv4l2/asciicam.c
new file mode 100644
index 0000000..5388967
--- /dev/null
+++ b/lib/libv4l2/asciicam.c
@@ -0,0 +1,63 @@
+/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
+   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
+
+gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
+ */
+#include <stdio.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+
+#include <linux/videodev2.h>
+
+#define SIZE 10*1024*1024
+
+char buf[SIZE];
+
+void main(void)
+{
+  int fd = v4l2_open("/dev/video2", O_RDWR);
+  int i;
+  static struct v4l2_format fmt;
+
+#if 1
+  fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+  fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
+  fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
+  fmt.fmt.pix.width = 640;
+  fmt.fmt.pix.height = 480;
+  if (fmt.fmt.pix.pixelformat != 'RGB3') /* v4l2_fourcc('R', 'G', 'B', '3'); */
+    printf("hmm. strange format?\n");
+  
+  printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
+#endif
+
+  for (i=0; i<500; i++) {
+    int num = v4l2_read(fd, buf, SIZE);
+    int y,x;
+    
+    printf("%d\n", num);
+#if 0
+    for (y = 0; y < 25; y++) {
+      for (x = 0; x < 80; x++) {
+	int y1 = y * 480/25;
+	int x1 = x * 640/80;
+	int c = buf[fmt.fmt.pix.width*3*y1 + 3*x1] +
+	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 1] +
+	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 2];
+
+	if (c < 30) c = ' ';
+	else if (c < 60) c = '.';
+	else if (c < 120) c = '_';
+	else if (c < 180) c = 'o';
+	else if (c < 300) c = 'x';
+	else if (c < 400) c = 'X';
+	else c = '#';
+	  
+	printf("%c", c);
+      }
+      printf("\n");
+    }
+#endif    
+  }
+}
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 343db5e..af740a7 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
 
@@ -70,6 +71,14 @@
 	} while (0)
 
 #define MIN(a, b) (((a) < (b)) ? (a) : (b))
+#define V4L2_MAX_SUBDEVS 16
+
+#define V4L2_MAX_FOCUS 10
+struct v4l2_focus_step {
+	int num;
+	int sharpness[V4L2_MAX_FOCUS];
+	int position[V4L2_MAX_FOCUS];
+};
 
 struct v4l2_dev_info {
 	int fd;
@@ -104,6 +113,14 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+        int subdev_fds[V4L2_MAX_SUBDEVS];
+  	int exposure; 
+  	int frame;
+  	int focus;
+
+	/* Autofocus parameters */
+	int focus_frame, focus_exposure, focus_step, sh_prev;
+	struct v4l2_focus_step focus_step_params;
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 0ba0a88..3b84437 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
 
@@ -100,6 +101,13 @@ static struct v4l2_dev_info devices[V4L2_MAX_DEVICES] = {
 };
 static int devices_used;
 
+#include "sdl.c"
+
+static struct sdl sdl;
+
+int v4l2_get_index(int fd);
+void my_main(void);
+
 static int v4l2_ensure_convert_mmap_buf(int index)
 {
 	if (devices[index].convert_mmap_buf != MAP_FAILED) {
@@ -311,7 +319,409 @@ static int v4l2_queue_read_buffer(int index, int buffer_index)
 	return 0;
 }
 
-static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
+static void v4l2_paint(char *buf, int x1, int y1, const struct v4l2_format *fmt)
+{
+  int y, x;
+  int width = fmt->fmt.pix.width;
+  printf("width = %d\n", width);
+    
+    for (y = 0; y < 25; y++) {
+      for (x = 0; x < 80; x++) {
+	int c = buf[width*4*y1 + 4*x1] +
+	  buf[width*4*y1 + 4*x1 + 1] +
+	  buf[width*4*y1 + 4*x1 + 2];
+
+	if (c < 30) c = ' ';
+	else if (c < 60) c = '.';
+	else if (c < 120) c = '_';
+	else if (c < 180) c = 'o';
+	else if (c < 300) c = 'x';
+	else if (c < 400) c = 'X';
+	else c = '#';
+	  
+	printf("%c", c);
+      }
+      printf("\n");
+    }
+}
+
+#define SX 80
+#define SY 25
+#define BUCKETS 20
+
+static void v4l2_histogram(unsigned char *buf, int cdf[], struct v4l2_format *fmt)
+{
+    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
+      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
+	pixel p = buf_pixel(fmt, buf, x, y);
+	
+	int b;
+	/* HACK: we divide green by 2 to have nice picture, undo it here. */
+	b = p.r + 2*p.g + p.b;
+	b = (b * BUCKETS)/(256);
+	cdf[b]++;
+      }
+}
+
+static long v4l2_sharpness(unsigned char *buf, struct v4l2_format *fmt)
+{
+  int h = fmt->fmt.pix.height;
+  int w = fmt->fmt.pix.width;
+  long r = 0;
+
+    for (int y = h/3; y < h-h/3; y+=h/9)
+      for (int x = w/3; x < w-w/3; x++) {
+	pixel p1 = buf_pixel(fmt, buf, x, y);
+	pixel p2 = buf_pixel(fmt, buf, x+2, y);
+	
+	int b1, b2;
+	/* HACK: we divide green by 2 to have nice picture, undo it here. */
+	b1 = p1.r + 2*p1.g + p1.b;
+	b2 = p2.r + 2*p2.g + p2.b;
+
+	int v;
+	v = (b1-b2)*(b1-b2);
+	if (v > 36)
+		r+=v;
+      }
+
+    return r;
+}
+
+int v4l2_set_exposure(int fd, int exposure)
+{
+	int index = v4l2_get_index(fd);
+
+	if (index == -1 || devices[index].convert == NULL) {
+		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
+		errno = EBADF;
+		return -1;
+	}
+
+	struct v4l2_control ctrl;
+	ctrl.id = V4L2_CID_EXPOSURE;
+	ctrl.value = exposure;
+	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
+	  printf("Could not set exposure\n");
+	}
+	return 0;
+}
+
+int v4l2_set_gain(int fd, int gain)
+{
+	int index = v4l2_get_index(fd);
+
+	if (index == -1 || devices[index].convert == NULL) {
+		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
+		errno = EBADF;
+		return -1;
+	}
+	
+	struct v4l2_control ctrl;
+	ctrl.id = 0x00980913;
+	ctrl.value = gain;
+	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
+	  printf("Could not set exposure\n");
+	}
+	return 0;
+}
+
+int v4l2_set_focus(int fd, int diopt)
+{
+	int index = v4l2_get_index(fd);
+
+	if (index == -1 || devices[index].convert == NULL) {
+		V4L2_LOG_ERR("v4l2_set_focus called with invalid fd: %d\n", fd);
+		errno = EBADF;
+		return -1;
+	}
+
+	struct v4l2_control ctrl;
+	ctrl.id = V4L2_CID_FOCUS_ABSOLUTE;
+	ctrl.value = diopt;
+	if (ioctl(devices[index].subdev_fds[1], VIDIOC_S_CTRL, &ctrl) < 0) {
+		printf("Could not set focus\n");
+	}
+	return 0;
+}
+
+#define LESS 1
+#define MID 0
+#define MORE 2
+
+static void v4l2_start_focus_step(struct v4l2_focus_step *fs, int focus, int step)
+{
+	int i;
+
+	fs->num = 3;
+	for (i=0; i<V4L2_MAX_FOCUS; i++) {
+		fs->sharpness[i] = -1;
+		fs->position[i] = -1;
+	}
+
+	fs->position[LESS] = focus - step;
+	fs->position[MID] = focus;
+	fs->position[MORE] = focus + step;
+}
+
+static int v4l2_focus_get_best(struct v4l2_focus_step *fs)
+{
+	int i, max = -1, maxi = -1;
+	
+	for (i=0; i<fs->num; i++)
+		if (max < fs->sharpness[i]) {
+			max = fs->sharpness[i];
+			maxi = i;
+		}
+
+	return maxi;
+}
+
+static void v4l2_auto_focus_continuous(struct v4l2_dev_info *m, int sh)
+{
+	int f = m->frame - m->focus_frame;
+	int step;
+	const int max_step = 300, min_step = 20;
+	struct v4l2_focus_step *fs = &m->focus_step_params;
+
+	if (m->focus_step == 0 || m->focus_step > max_step) {
+		printf("step reset -- max\n");
+		m->focus_step = max_step;
+	}
+	if (m->focus_step < min_step) {
+		printf("step reset -- 10 (%d)\n", m->focus_step);
+		m->focus_step = min_step;
+		/* It takes cca 5.7 seconds to achieve the focus:
+		   0.76user 0.30system 5.66 (0m5.661s) elapsed 18.72%CPU
+		 */
+		printf("Focused at %d\n", m->focus);
+		exit(0);
+	}
+
+	step = m->focus_step;
+
+	if (m->exposure != m->focus_exposure) {
+		m->focus_frame = m->frame;
+		m->focus_exposure = m->exposure;
+		v4l2_start_focus_step(fs, m->focus, m->focus_step);
+		return;
+	}
+	if (m->focus < step) {
+		m->focus = step;
+	}
+
+	const int every = 3;
+	if (f%every)
+		return;
+
+	{
+		int i = f/every;
+
+		if (i == 0) {
+			printf("Can not happen?\n");
+			return;
+		}
+		i--;
+		if (i < fs->num)
+			v4l2_set_focus(m->fd, fs->position[i]);
+		if (i > 0)
+			fs->sharpness[i-1] = sh;
+		if (i < fs->num)
+			return;
+	}
+	int i;
+	for (i=0; i<fs->num; i++) {
+		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
+	}
+	int best = v4l2_focus_get_best(fs);
+	if ((fs->sharpness[best] < m->sh_prev) && (m->focus_step < max_step)) {
+		m->focus_step *=2;
+		m->sh_prev = m->sh_prev * 0.9;
+		printf("step up %d\n", m->focus_step);
+	} else if (best == LESS) {
+		printf("less ");
+		m->focus -= step;
+	} else if (best == MORE) {
+		printf("more ");
+		m->focus += step;
+	} else {
+		m->sh_prev = fs->sharpness[MID];
+		m->focus_step = m->focus_step / 2;
+		printf("step %d ", m->focus_step);
+	}
+	m->focus_frame = m->frame;
+	v4l2_start_focus_step(fs, m->focus, m->focus_step);
+	printf("Focus now %d\n", m->focus);
+}
+
+static void v4l2_start_focus_sweep(struct v4l2_focus_step *fs, int focus, int step)
+{
+	int i;
+
+	fs->num = V4L2_MAX_FOCUS;
+	for (i=0; i<V4L2_MAX_FOCUS; i++) {
+		fs->sharpness[i] = -1;
+		fs->position[i] = -1;
+	}
+
+	int f = focus;
+	for (i=0; i<V4L2_MAX_FOCUS; i++) {
+		fs->position[i] = f;
+		f += step;
+	}
+}
+
+static void v4l2_auto_focus_single(struct v4l2_dev_info *m, int sh)
+{
+	int f = m->frame - m->focus_frame;
+	int step;
+	struct v4l2_focus_step *fs = &m->focus_step_params;
+
+	if (m->focus_step == 0) {
+		printf("step reset -- max\n");
+		m->focus_step = 1;
+	}
+
+	if (m->exposure != m->focus_exposure) {
+		m->focus_frame = m->frame;
+		m->focus_exposure = m->exposure;
+		v4l2_start_focus_sweep(fs, 0, 100);
+		return;
+	}
+
+	const int every = 3;
+	if (f%every)
+		return;
+
+	{
+		int i = f/every;
+
+		if (i == 0) {
+			printf("Can not happen?\n");
+			return;
+		}
+		i--;
+		if (i < fs->num)
+			v4l2_set_focus(m->fd, fs->position[i]);
+		if (i > 0)
+			fs->sharpness[i-1] = sh;
+		if (i < fs->num)
+			return;
+	}
+#if 0
+	int i;
+	for (i=0; i<fs->num; i++) {
+		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
+	}
+#endif
+	int best = v4l2_focus_get_best(fs);
+	m->focus_frame = m->frame;
+	switch (m->focus_step) {
+	case 1:
+		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
+		v4l2_start_focus_sweep(fs, fs->position[best] - 50, 10);
+		m->focus_step = 2;
+		break;
+	case 2:
+		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
+		v4l2_start_focus_sweep(fs, fs->position[best] - 10, 2);
+		m->focus_step = 3;
+		break;
+	case 3:
+		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
+		printf("done.\n");
+		exit(0);
+	}
+}
+
+
+static void v4l2_auto_exposure(int index, struct v4l2_buffer *buf)
+{
+	struct v4l2_format *fmt;
+	int cdf[BUCKETS] = { 0, };
+	int i;
+
+	fmt = &devices[index].src_fmt;
+
+	v4l2_histogram(devices[index].frame_pointers[buf->index], cdf, fmt);
+
+#if 0
+	printf("hist: ");
+	for (i = 0; i<BUCKETS; i++)
+		printf("%d ", cdf[i]);
+	printf("\n");
+#endif
+	for (i=1; i<BUCKETS; i++)
+		cdf[i] += cdf[i-1];
+
+	int b = BUCKETS;
+	int brightPixels = cdf[b-1] - cdf[b-8];
+	int targetBrightPixels = cdf[b-1]/50;
+	int maxSaturatedPixels = cdf[b-1]/200;
+	int saturatedPixels = cdf[b-1] - cdf[b-2];
+	// how much should I change brightness by
+	float adjustment = 1.0f;
+#if 0
+	printf( "AutoExposure: totalPixels: %d,"
+		"brightPixels: %d, targetBrightPixels: %d,"
+		"saturatedPixels: %d, maxSaturatedPixels: %d\n",
+		cdf[b-1], brightPixels, targetBrightPixels,
+		saturatedPixels, maxSaturatedPixels);
+#endif
+	  
+	if (saturatedPixels > maxSaturatedPixels) {
+		// first don't let things saturate too much
+		adjustment = 1.0f - ((float)(saturatedPixels - maxSaturatedPixels))/cdf[b-1];
+	} else if (brightPixels < (targetBrightPixels - (saturatedPixels * 4))) {
+		// increase brightness to try and hit the desired number of well exposed pixels
+		int l = b-6;
+		while (brightPixels < targetBrightPixels && l > 0) {
+			brightPixels += cdf[l];
+			brightPixels -= cdf[l-1];
+			l--;
+		}
+
+		// that level is supposed to be at b-11;
+		adjustment = ((float) (b-6+1))/(l+1);
+	} else {
+		// we're not oversaturated, and we have enough bright pixels. Do nothing.
+	}
+
+	{
+		float limit = 4;
+		if (adjustment > limit) { adjustment = limit; }
+		if (adjustment < 1/limit) { adjustment = 1/limit; }
+	}
+	  
+	if (!devices[index].exposure)
+		devices[index].exposure = 1;
+	devices[index].exposure *= adjustment;
+	if (adjustment != 1.)
+		printf( "AutoExposure: adjustment: %f exposure %d\n", adjustment, devices[index].exposure);
+
+	v4l2_set_exposure(devices[index].fd, devices[index].exposure);
+}
+
+static void v4l2_statistics(int index, struct v4l2_buffer *buf)
+{
+	unsigned char *b;
+	struct v4l2_format *fmt;
+
+	fmt = &devices[index].src_fmt;
+	b = devices[index].frame_pointers[buf->index];
+
+	if (!(devices[index].frame%3))
+		v4l2_auto_exposure(index, buf);
+	
+	int sh = v4l2_sharpness(b, fmt);
+	v4l2_auto_focus_single(&devices[index], sh);
+	
+	devices[index].frame++;
+	if (!(devices[index].frame%4))
+		sdl_render(&sdl, b, fmt);
+}
+
+int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
 		unsigned char *dest, int dest_size)
 {
 	const int max_tries = V4L2_IGNORE_FIRST_FRAME_ERRORS + 1;
@@ -345,6 +755,13 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
 			errno = -EINVAL;
 			return -1;
 		}
+		
+#if 1
+		v4l2_statistics(index, buf);
+#endif
+#if 0
+		/* This is rather major eater of CPU time. CPU time goes from 80% to 4% 
+		   when conversion is disabled. */
 
 		result = v4lconvert_convert(devices[index].convert,
 				&devices[index].src_fmt, &devices[index].dest_fmt,
@@ -352,7 +769,7 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
 				buf->bytesused, dest ? dest : (devices[index].convert_mmap_buf +
 					buf->index * devices[index].convert_mmap_frame_size),
 				dest_size);
-
+#endif
 		if (devices[index].first_frame) {
 			/* Always treat convert errors as EAGAIN during the first few frames, as
 			   some cams produce bad frames at the start of the stream
@@ -789,18 +1206,24 @@ no_capture:
 
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
-	   a fram erate using the S_PARM ioctl after a S_FMT */
+	   a frame rate using the S_PARM ioctl after a S_FMT */
 	if (devices[index].convert)
 		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
 	v4l2_update_fps(index, &parm);
 
+	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
+	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
+
+	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
+	       devices[index].subdev_fds[1]);
+
 	V4L2_LOG("open: %d\n", fd);
 
 	return fd;
 }
 
 /* Is this an fd for which we are emulating v4l1 ? */
-static int v4l2_get_index(int fd)
+int v4l2_get_index(int fd)
 {
 	int index;
 
@@ -823,6 +1246,10 @@ int v4l2_close(int fd)
 {
 	int index, result;
 
+	if (fd == -2) {
+	  my_main();
+	}
+
 	index = v4l2_get_index(fd);
 	if (index == -1)
 		return SYS_CLOSE(fd);
@@ -1782,3 +2209,65 @@ int v4l2_get_control(int fd, int cid)
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }
+
+void v4l2_debug(void)
+{
+	printf("debug\n");
+}
+
+/* ------------------------------------------------------------------ */
+
+#define SIZE 10*1024*1024
+
+char buf[SIZE];
+
+void my_main(void)
+{
+	int fd = v4l2_open("/dev/video2", O_RDWR);
+	int i;
+	static struct v4l2_format fmt;
+
+	fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
+	fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
+	fmt.fmt.pix.width = 640;
+	fmt.fmt.pix.height = 480;
+
+	v4l2_set_gain(fd, 300);
+	v4l2_set_exposure(fd, 10000);
+	v4l2_set_focus(fd, 0);
+
+
+	printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
+
+	printf("capture is %d, %d\n", fmt.fmt.pix.width, fmt.fmt.pix.height);
+	/* factor == 2 still fits window, but very slow. factor == 6 .. */
+	/* factor needs to be odd, otherwise ... fun with BA10 format. */
+	sdl_init(&sdl, fmt.fmt.pix.width, fmt.fmt.pix.height, 5);
+  
+	v4l2_debug();
+
+	/* In 800x600 "raw" mode, this should take cca 17.8 seconds (without
+	   sdl output. CPU usage should be cca 5% without conversion). That's 28 fps.
+	   (benchmark with i<500)
+	*/
+	for (i=0; i<50000; i++) {
+		int num = v4l2_read(fd, buf, SIZE);
+
+		if (i==490) {
+			printf("Focus to closest.... -------------------\n");
+			v4l2_set_focus(fd, 99999);
+		}
+    
+#if 0
+		v4l2_paint(buf, 640/80, 480/25, &fmt);
+#endif
+		/* Over USB connection, rendering every single frame slows
+		   execution down from 23 seconds to 36 seconds. */
+#if 0
+		if (!(i%4))
+			sdl_render(&sdl, buf, &fmt);
+#endif
+	}
+  
+}
diff --git a/lib/libv4l2/sdl.c b/lib/libv4l2/sdl.c
new file mode 100644
index 0000000..17a8d24
--- /dev/null
+++ b/lib/libv4l2/sdl.c
@@ -0,0 +1,530 @@
+/* -*- c-file-style: "linux" -*- */
+/* SDL support.
+
+   Copyright 2017 Pavel Machek, LGPL
+*/
+
+#include <SDL2/SDL.h>
+#include <SDL2/SDL_image.h>
+
+struct sdl {
+	SDL_Window *window;
+	SDL_Surface *liveview, *screen;
+
+	int wx, wy;
+	int sx, sy;
+	int bx, by;
+	int factor;
+	float focus, gain, exposure, do_focus, do_exposure; 
+};
+
+#if 0
+void loop(void) {
+	int done;
+	SDL_Event event;
+
+	while(!done){ //While program isn't done                                   
+		while(SDL_PollEvent(&event)){ //Poll events                        
+			switch(event.type){ //Check event type                     
+			case SDL_QUIT: //User hit the X (or equivelent)            
+				done = true; //Make the loop end                   
+				break; //We handled the event                      
+			} //Finished with current event                            
+		} //Done with all events for now                                   
+	} //Program done, exited                                                   
+}
+#endif
+
+typedef struct {
+	Uint8 r;
+	Uint8 g;
+	Uint8 b;
+	Uint8 alpha;
+} pixel;
+
+#define d_raw 1
+
+void sfc_put_pixel(SDL_Surface* liveview, int x, int y, pixel *p)
+{
+	Uint32* p_liveview = (Uint32*)liveview->pixels;
+	p_liveview += y*liveview->w+x;
+	*p_liveview = SDL_MapRGBA(liveview->format,p->r,p->g,p->b,p->alpha);
+}
+
+#if 0
+int pix_exposure(float x)
+{
+	return sy*x / 199410.0;
+}
+
+int pix_gain(float x)
+{
+	return sy*x / 16.0;
+}
+
+int render_statistics(SDL_Surface* liveview)
+{
+	pixel white;
+	white.r = (Uint8)0xff;
+	white.g = (Uint8)0xff;
+	white.b = (Uint8)0xff;
+	white.alpha = (Uint8)128;
+
+	//printf("Stats: focus %d, gain %d, exposure %d\n", focus, gain, exposure);
+
+	for (int x=0; x<sx && x<sx*focus; x++)
+		put_pixel(liveview, x, 0, &white);
+
+	for (int y=0; y<sy && y<pix_gain(gain); y++)
+		put_pixel(liveview, 0, y, &white);
+
+	for (int y=0; y<sy && y<pix_exposure(exposure); y++)
+		put_pixel(liveview, sx-1, y, &white);
+
+	for (int x=0; x<sx; x++)
+		put_pixel(liveview, x, sy-1, &white);
+
+	return 0;
+}
+#endif
+
+void sdl_begin_paint(struct sdl *m) {
+	//Fill the surface white                                                   
+	SDL_FillRect(m->liveview, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+
+	SDL_LockSurface(m->liveview);
+}
+
+void sdl_finish_paint(struct sdl *m) {
+	SDL_UnlockSurface(m->liveview);
+	SDL_Rect rcDest = { m->bx, m->by, m->sx, m->sy };
+
+	SDL_BlitSurface(m->liveview, NULL, m->screen, &rcDest);
+	//Update the surface                                                       
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+  
+void sdl_paint_image(struct sdl *m, char **xpm, int x, int y) {
+	SDL_Surface *image = IMG_ReadXPMFromArray(xpm);
+	if (!image) {
+		printf("IMG_Load: %s\n", IMG_GetError());
+		exit(1);
+	}
+
+	int x_pos = x - image->w/2, y_pos = y - image->h/2;
+
+	SDL_Rect rcDest = { x_pos, y_pos, image->w, image->h };
+	int r = SDL_BlitSurface ( image, NULL, m->screen, &rcDest );
+
+	if (r) {
+		printf("Error blitting: %s\n", SDL_GetError());
+		exit(1);
+	}
+	SDL_FreeSurface ( image );
+}
+
+void sdl_paint_ui(struct sdl *m) {
+	static char *wait_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"....########....",
+		".....#....#.....",
+		".....#....#.....",
+		"......#..#......",
+		".......##.......",
+		"......#..#......",
+		".....#....#.....",
+		".....#....#.....",
+		"....########....",
+	};
+
+	static char *ok_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"...............#",
+		"............###.",
+		"..........##....",
+		"#.......##......",
+		".#.....#........",
+		"..#...#.........",
+		"..#..#..........",
+		"...##...........",
+		"...#............",
+	};
+
+	static char *exit_xpm[] = {
+		"16 9 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"....x......x....",
+		".....x....x.....",
+		"......x..x......",
+		".......xx.......",
+		".......xx.......",
+		"......x..x......",
+		".....x....x.....",
+		"....x......x....",
+		"................",
+	};
+
+	static char *f1m_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"....##..........",
+		"...#.#..........",
+		"..#..#..........",
+		".....#...#.#.##.",
+		".....#...##.#..#",
+		".....#...#..#..#",
+		".....#...#..#..#",
+		".....#...#..#..#",
+		"................",
+	};
+
+	static char *f25cm_xpm[] = {
+		"16 9 2 1",
+		"x c #ffffff",
+		". c #000000",
+		".xxx..xxxx......",
+		"x...x.x.........",
+		"...x..xxx.......",
+		"..x......x..xx.x",
+		".x.......x.x.xxx",
+		"xxxxx.xxx...xxxx",
+		"................",
+		"................",
+		"................",
+	};
+
+	static char *iso400_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"x..x.xxxx.xxxx..",
+		"x..x.x..x.x..x..",
+		"xxxx.x..x.x..x..",
+		"...x.x..x.x..x..",
+		"...x.xxxx.xxxx..",
+		"................",
+		".x..xx..x.......",
+		".x.x...x.x......",
+		".x..x..x.x......",
+		".x...x.x.x......",
+		".x.xx...x.......",
+		"................",
+	};
+
+	static char *time_1_100_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		".x....x.........",
+		".x...x..........",
+		".x..x...........",
+		".x.x............",
+		"................",
+		"..x.xxxx.xxxx...",
+		"..x.x..x.x..x...",
+		"..x.x..x.x..x...",
+		"..x.x..x.x..x...",
+		"..x.x..x.x..x...",
+		"..x.xxxx.xxxx...",
+		"................",
+	};
+
+	static char *time_1_10_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		".x....x.........",
+		".x...x..........",
+		".x..x...........",
+		".x.x............",
+		"................",
+		"..x.xxxx........",
+		"..x.x..x........",
+		"..x.x..x........",
+		"..x.x..x........",
+		"..x.x..x........",
+		"..x.xxxx........",
+		"................",
+	};
+
+	static char *af_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..x..........",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+
+	static char *ae_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..xxxxxxx....",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+    
+	static char *not_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"......xxxxx.....",
+		"....xx.....xx...",
+		"...x.........x..",
+		"..x........xx.x.",
+		"..x......xx...x.",
+		".x.....xx......x",
+		".x...xx........x",
+		"..xxx.........x.",
+		"..x...........x.",
+		"...x.........x..",
+		"....xx.....xx...",
+		"......xxxxx.....",
+	};
+
+	static char *empty_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+
+	SDL_FillRect(m->screen, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+	sdl_paint_image(m, wait_xpm,  m->wx/2,     m->wy/2);
+	sdl_paint_image(m, ok_xpm,    m->wx-m->bx/2,  m->wy-m->by/2);
+	sdl_paint_image(m, exit_xpm,  m->bx/2,     m->wy-m->by/2);
+	sdl_paint_image(m, f1m_xpm,   m->bx+m->sx/20, m->by/2);
+	sdl_paint_image(m, f25cm_xpm, m->bx+m->sx/5,  m->by/2);
+
+	sdl_paint_image(m, af_xpm,    m->bx/2,     m->by/2);
+	if (!m->do_focus) {
+		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
+	}
+	sdl_paint_image(m, ae_xpm,    m->wx-m->bx/2,  m->by/2);
+	if (!m->do_exposure) {
+		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
+	}
+
+#if 0
+	sdl_paint_image(m, time_1_100_xpm, m->wx-m->bx/2, m->by+pix_exposure(10000));
+	sdl_paint_image(m, time_1_10_xpm,  m->wx-m->bx/2, m->by+pix_exposure(100000));
+	sdl_paint_image(m, iso400_xpm,     m->bx/2,    m->by+pix_gain(4.0));
+#endif
+		
+	SDL_UpdateWindowSurface(m->window);
+}
+
+void fmt_print(struct v4l2_format *fmt)
+{
+	int f;
+	printf("Format: %dx%d. ", fmt->fmt.pix.width, fmt->fmt.pix.height);
+	printf("%x ", fmt->fmt.pix.pixelformat);
+	f = fmt->fmt.pix.pixelformat;
+	for (int i=0; i<4; i++) {
+		printf("%c", f & 0xff);
+		f >>= 8;
+	}
+	printf("\n");
+}
+
+pixel buf_pixel(struct v4l2_format *fmt, unsigned char *buf, int x, int y)
+{
+	pixel p = { 0, 0, 0, 0 };
+	int pos = x + y*fmt->fmt.pix.width;
+	int b;
+
+	p.alpha = 128;
+
+	switch (fmt->fmt.pix.pixelformat) {
+	case '01AB': /* BA10, aka GRBG10, 
+			https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-srggb10.html?highlight=ba10
+		     */
+		b = buf[pos*2];
+		b += buf[pos*2+1] << 8;
+		b /= 4;
+		if ((y % 2) == 0) {
+			switch (x % 2) {
+			case 0: p.g = b/2; break;
+			case 1: p.r = b; break;
+			}
+		} else {
+			switch (x % 2) {
+			case 0: p.b = b; break;
+			case 1: p.g = b/2; break;
+			}
+		}
+		break;
+
+	case V4L2_PIX_FMT_RGB24:
+		pos *= 4;
+		p.r = buf[pos];
+		p.g = buf[pos+1];
+		p.b = buf[pos+2];
+		break;
+
+	default:
+		printf("Wrong pixel format!\n");
+		fmt_print(fmt);
+		exit(1);
+	}
+
+	return p;
+}
+
+void sdl_render(struct sdl *m, unsigned char *buf, struct v4l2_format *fmt)
+{
+	if (!m->window) 
+		return;
+	sdl_begin_paint(m);    
+	/* do your rendering here */
+
+	for (int y = 0; y < m->sy; y++)
+		for (int x = 0; x < m->sx; x++) {
+			pixel p = buf_pixel(fmt, buf, x*m->factor, y*m->factor);
+			p.alpha = 128;
+			sfc_put_pixel(m->liveview, x, y, &p);
+		}
+
+	//render_statistics(liveview);
+
+	sdl_finish_paint(m);
+}
+
+#if 0
+void imageStatistics(FCam::Image img)
+{
+	int sx, sy;
+
+	sx = img.size().width;                                                          
+	sy = img.size().height;                                                         
+	printf("Image is %d x %d, ", sx, sy);                                           
+
+	printf("(%d) ", img.brightness(sx/2, sy/2));
+
+	int dark = 0;                                                                   
+	int bright = 0;                                                                 
+	int total = 0;                                                                  
+#define RATIO 10
+	for (int y = sy/(2*RATIO); y < sy; y += sy/RATIO)                               
+		for (int x = sx/(2*RATIO); x < sx; x += sx/RATIO) {                    
+			int br = img.brightness(x, y);                               
+
+			if (!d_raw) {
+				/* It seems real range is 60 to 218 */
+				if (br > 200)
+					bright++;                                               
+				if (br < 80)
+					dark++;
+			} else {
+				/* there's a lot of noise, it seems black is commonly 65..71,
+				   bright is cca 1023 */
+				if (br > 1000)
+					bright++;
+				if (br < 75)
+					dark++;
+			}
+			total++;                                                      
+		}
+
+	printf("%d dark %d bri,", dark, bright);                     
+
+	long sharp = 0;
+	for (int y = sy/3; y < 2*(sy/3); y+=sy/12) {
+		int b = -1;
+		for (int x = sx/3; x < 2*(sx/3); x++) {
+			int b2;                                                                
+			b2 = img.brightness(x, y/2);                                          
+			if (b != -1)
+				sharp += (b-b2) * (b-b2);                                              
+			b = b2;                                                                
+		}
+	}
+	printf("sh %d\n", sharp);
+}
+#endif
+
+
+void sdl_init(struct sdl *m, int _sx, int _sy, int _factor)
+{
+	printf("Initing SDL\n");
+
+	if (SDL_Init(SDL_INIT_VIDEO) < 0) {
+		printf("Could not init SDL\n");
+		exit(1);
+	}
+
+	atexit(SDL_Quit);
+
+	m->wx = 800;
+	m->wy = 400;
+
+	m->window = SDL_CreateWindow( "Camera", SDL_WINDOWPOS_UNDEFINED,
+				      SDL_WINDOWPOS_UNDEFINED, m->wx, m->wy,
+				      SDL_WINDOW_SHOWN );
+	if (m->window == NULL) {
+		printf( "Window could not be created! SDL_Error: %s\n", SDL_GetError() );
+	}
+
+	m->screen = SDL_GetWindowSurface(m->window);
+	if (!m->screen) {
+		printf("Couldn't create liveview\n");
+		exit(1);
+	}
+
+	m->sx = _sx;
+	m->sy = _sy;
+	m->factor = _factor;
+
+	m->sx /= m->factor;
+	m->sy /= m->factor;
+    
+	m->focus = 0;
+	m->gain = 0;
+	m->exposure = 0;
+
+	m->bx = (m->wx-m->sx)/2;
+	m->by = (m->wy-m->sy)/2;
+
+	m->liveview = SDL_CreateRGBSurface(0,m->sx,m->sy,32,0,0,0,0);
+	if (!m->liveview) {
+		printf("Couldn't create liveview\n");
+		exit(1);
+	}
+	sdl_paint_ui(m);
+}
diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
index d3d8936..7521ec8 100644
--- a/lib/libv4lconvert/libv4lconvert.c
+++ b/lib/libv4lconvert/libv4lconvert.c
@@ -1205,6 +1205,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			v4lconvert_swap_uv(src, dest, fmt);
 			break;
 		}
+		default:
+		  /* This is bad, fixme? */
 		break;
 
 	case V4L2_PIX_FMT_YVU420:
@@ -1228,6 +1230,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 		case V4L2_PIX_FMT_YVU420:
 			memcpy(dest, src, width * height * 3 / 2);
 			break;
+		default:
+		  /* This is bad, fixme? */
 		}
 		break;
 
diff --git a/utils/decode_tm6000/Makefile.am b/utils/decode_tm6000/Makefile.am
index ac4e85e..f059e3c 100644
--- a/utils/decode_tm6000/Makefile.am
+++ b/utils/decode_tm6000/Makefile.am
@@ -1,4 +1,4 @@
 bin_PROGRAMS = decode_tm6000
 decode_tm6000_SOURCES = decode_tm6000.c
-decode_tm6000_LDADD = ../libv4l2util/libv4l2util.la
+decode_tm6000_LDADD = ../libv4l2/libv4l2.la
 decode_tm6000_LDFLAGS = $(ARGP_LIBS)
diff --git a/utils/decode_tm6000/decode_tm6000.c b/utils/decode_tm6000/decode_tm6000.c
index 4bffbdd..fda7e3b 100644
--- a/utils/decode_tm6000/decode_tm6000.c
+++ b/utils/decode_tm6000/decode_tm6000.c
@@ -1,365 +1,16 @@
-/*
-   decode_tm6000.c - decode multiplexed format from TM5600/TM6000 USB
+/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
+   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
 
-   Copyright (C) 2007 Mauro Carvalho Chehab <mchehab@infradead.org>
-
-   This program is free software; you can redistribute it and/or modify
-   it under the terms of the GNU General Public License as published by
-   the Free Software Foundation version 2.
-
-   This program is distributed in the hope that it will be useful,
-   but WITHOUT ANY WARRANTY; without even the implied warranty of
-   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-   GNU General Public License for more details.
-
-   You should have received a copy of the GNU General Public License
-   along with this program; if not, write to the Free Software
-   Foundation, Inc., 51 Franklin Street, Suite 500, Boston, MA 02110-1335 USA.
+gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
  */
-#include "../libv4l2util/v4l2_driver.h"
 #include <stdio.h>
-#include <string.h>
-#include <argp.h>
-#include <unistd.h>
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <fcntl.h>
-#include <stdlib.h>
-#include <errno.h>
-
-const char *argp_program_version="decode_tm6000 version 0.0.1";
-const char *argp_program_bug_address="Mauro Carvalho Chehab <mchehab@infradead.org>";
-const char doc[]="Decodes tm6000 proprietary format streams";
-const struct argp_option options[] = {
-	{"verbose",	'v',	0,	0,	"enables debug messages", 0},
-	{"device",	'd',	"DEV",	0,	"uses device for reading", 0},
-	{"output",	'o',	"FILE",	0,	"outputs raw stream to a file", 0},
-	{"input",	'i',	"FILE",	0,	"parses a file, instead of a device", 0},
-	{"freq",	'f',	"Freq",	0,	"station frequency, in MHz (default is 193.25)", 0},
-	{"nbufs",	'n',	"quant",0,	"number of video buffers", 0},
-	{"audio",	'a',	0,	0,	"outputs audio on stdout", 0},
-	{"read",	'r',	0,	0,	"use read() instead of mmap method", 0},
-	{ 0, 0, 0, 0, 0, 0 }
-};
-
-static char outbuf[692224];
-static int debug=0, audio=0, use_mmap=1, nbufs=4;
-static float freq_mhz=193.25;
-static char *devicename="/dev/video0";
-static char *filename=NULL;
-static enum {
-	NORMAL,
-	INPUT,
-	OUTPUT
-} mode = NORMAL;
-
-static FILE *fout;
-
-//const char args_doc[]="ARG1 ARG2";
-
-static error_t parse_opt (int key, char *arg, struct argp_state *state)
-{
-	switch (key) {
-	case 'a':
-		audio++;
-		break;
-	case 'r':
-		use_mmap=0;
-		break;
-	case 'v':
-		debug++;
-		break;
-	case 'd':
-		devicename=arg;
-		break;
-	case 'i':
-	case 'o':
-		if (mode!=NORMAL) {
-			argp_error(state,"You can't use input/output options simultaneously.\n");
-			break;
-		}
-		if (key=='i')
-			mode=INPUT;
-		else
-			mode=OUTPUT;
-
-		filename=arg;
-		break;
-	case 'f':
-		freq_mhz=atof(arg);
-		break;
-	case 'n':
-		nbufs=atoi(arg);
-		if  (nbufs<2)
-			nbufs=2;
-		break;
-	default:
-		return ARGP_ERR_UNKNOWN;
-	}
-	return 0;
-}
-
-static struct argp argp = {
-	.options=options,
-	.parser=parse_opt,
-	.args_doc=NULL,
-	.doc=doc,
-};
 
-#define TM6000_URB_MSG_LEN 180
-enum {
-	TM6000_URB_MSG_VIDEO=1,
-	TM6000_URB_MSG_AUDIO,
-	TM6000_URB_MSG_VBI,
-	TM6000_URB_MSG_PTS,
-	TM6000_URB_MSG_ERR,
-};
-
-const char *tm6000_msg_type[]= {
-	"unknown(0)",	//0
-	"video",	//1
-	"audio",	//2
-	"vbi",		//3
-	"pts",		//4
-	"err",		//5
-	"unknown(6)",	//6
-	"unknown(7)",	//7
-};
-
-#define dprintf(fmt,arg...) \
-	if (debug) fprintf(stderr, fmt, ##arg)
-
-static int recebe_buffer (struct v4l2_buffer *v4l2_buf, struct v4l2_t_buf *buf)
-{
-	dprintf("Received %zd bytes\n", buf->length);
-fflush(stdout);
-	memcpy (outbuf,buf->start,buf->length);
-	return buf->length;
-}
-
-
-static int prepare_read (struct v4l2_driver *drv)
-{
-	struct v4l2_format fmt;
-	double freq;
-	int rc;
-
-	memset (drv,0,sizeof(*drv));
-
-	if (v4l2_open (devicename, 1,drv)<0) {
-		perror ("Error opening dev");
-		return -1;
-	}
-
-	memset (&fmt,0,sizeof(fmt));
-
-	uint32_t pixelformat=V4L2_PIX_FMT_TM6000;
-
-	if (v4l2_gettryset_fmt_cap (drv,V4L2_SET,&fmt, 720, 480,
-				    pixelformat,V4L2_FIELD_ANY)) {
-		perror("set_input to tm6000 raw format");
-		return -1;
-	}
-
-	if (freq_mhz) {
-		freq=freq_mhz * 1000 * 1000;
-		rc=v4l2_getset_freq (drv,V4L2_SET, &freq);
-		if (rc<0)
-			printf ("Cannot set freq to %.3f MHz\n",freq_mhz);
-	}
-
-	if (use_mmap) {
-		printf("Preparing for receiving frames on %d buffers...\n",nbufs);
-		fflush (stdout);
-
-		rc=v4l2_mmap_bufs(drv, nbufs);
-		if (rc<0) {
-			printf ("Cannot mmap %d buffers\n",nbufs);
-			return -1;
-		}
-
-//		v4l2_stop_streaming(&drv);
-		rc=v4l2_start_streaming(drv);
-		if (rc<0) {
-			printf ("Cannot start streaming\n");
-			return -1;
-		}
-	}
-	printf("Waiting for frames...\n");
-
-	return 0;
-}
+#include <linux/videodev2.h>
 
-static int read_stream (struct v4l2_driver *drv, int fd)
+void main(void)
 {
-	if (use_mmap) {
-		fd_set fds;
-		struct timeval tv;
-		int r;
-
-		FD_ZERO (&fds);
-		FD_SET (fd, &fds);
-
-		/* Timeout. */
-		tv.tv_sec = 2;
-		tv.tv_usec = 0;
-
-		r = select (fd + 1, &fds, NULL, NULL, &tv);
-		if (-1 == r) {
-			if (EINTR == errno) {
-				perror ("select");
-				return -errno;
-			}
-		}
-
-		if (0 == r) {
-			fprintf (stderr, "select timeout\n");
-			return -errno;
-		}
-
-		return v4l2_rcvbuf(drv, recebe_buffer);
-	} else {
-		int size=read(fd, outbuf, sizeof(outbuf));
-		return size;
-	}
-
-	return 0;
-}
-
-static int read_char (struct v4l2_driver *drv, int fd)
-{
-	static int sizebuf=0;
-	static unsigned char *p=NULL;
-	unsigned char c;
-
-	if (sizebuf<=0) {
-		sizebuf=read_stream(drv,fd);
-		if (sizebuf<=0)
-			return -1;
-		p=(unsigned char *)outbuf;
-	}
-	c=*p;
-	p++;
-	sizebuf--;
-
-	return c;
-}
-
-
-int main (int argc, char*argv[])
-{
-	int fd;
-	unsigned int i;
-	unsigned char buf[TM6000_URB_MSG_LEN], img[720*2*480];
-	unsigned int  cmd, size, field, block, line, pos=0;
-	unsigned long header=0;
-	int           linesize=720*2,skip=0;
-	struct v4l2_driver drv;
-
-	argp_parse (&argp, argc, argv, 0, 0, 0);
-
-	if (mode!=INPUT) {
-		if (prepare_read (&drv)<0)
-			return -1;
-		fd=drv.fd;
-	} else {
-		/*mode == INPUT */
-
-		fd=open(filename,O_RDONLY);
-		if (fd<0) {
-			perror ("error opening a file for parsing");
-			return -1;
-		}
-		dprintf("file %s opened for parsing\n",filename);
-		use_mmap=0;
-	}
-
-	if (mode==OUTPUT) {
-		fout=fopen(filename,"w");
-		if (!fout) {
-			perror ("error opening a file to write");
-			return -1;
-		}
-		dprintf("file %s opened for output\n",filename);
-
-		do {
-			size=read_stream (&drv,fd);
-
-			if (size<=0) {
-				close (fd);
-				return -1;
-			}
-			dprintf("writing %d bytes\n",size);
-			fwrite(outbuf,1, size,fout);
-//			fflush(fout);
-		} while (1);
-	}
-
-
-	while (1) {
-		skip=0;
-		header=0;
-		do {
-			int c;
-			c=read_char (&drv,fd);
-			if (c<0) {
-				perror("read");
-				return -1;
-			}
-
-			header=(header>>8)&0xffffff;
-			header=header|(c<<24);
-			skip++;
-		} while ( (((header>>24)&0xff) != 0x47) );
-
-		/* split the header fields */
-		size  = (((header & 0x7e)<<1) -1) * 4;
-		block = (header>>7) & 0xf;
-		field = (header>>11) & 0x1;
-		line  = (header>>12) & 0x1ff;
-		cmd   = (header>>21) & 0x7;
-
-		/* Read the remaining buffer */
-		for (i=0;i<sizeof(buf);i++) {
-			int c;
-			c=read_char (&drv,fd);
-			if (c<0) {
-				perror("read");
-				return -1;
-			}
-			buf[i]=c;
-		}
-
-		/* FIXME: Mounts the image as field0+field1
-			* It should, instead, check if the user selected
-			* entrelaced or non-entrelaced mode
-			*/
-		pos=((line<<1)+field)*linesize+
-					block*TM6000_URB_MSG_LEN;
-
-			/* Prints debug info */
-		dprintf("0x%08x (skip %d), %s size=%d, line=%d, field=%d, block=%d\n",
-				(unsigned int)header, skip,
-				 tm6000_msg_type[cmd],
-				 size, line, field, block);
-
-		/* Don't allow to write out of the buffer */
-		if (pos+sizeof(buf) > sizeof(img))
-			cmd = TM6000_URB_MSG_ERR;
-
-		/* handles each different URB message */
-		switch(cmd) {
-		case TM6000_URB_MSG_VIDEO:
-			/* Fills video buffer */
-			memcpy(buf,&img[pos],sizeof(buf));
-		case TM6000_URB_MSG_AUDIO:
-			if (audio)
-				fwrite(buf,sizeof(buf),1,stdout);
-//		case TM6000_URB_MSG_VBI:
-//		case TM6000_URB_MSG_PTS:
-		break;
-		}
-	}
-	close(fd);
-	return 0;
+  v4l2_close(-2);
 }

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Hi Pavel,

Em Mon, 24 Apr 2017 11:30:59 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> For focus to be useful, we need autofocus implmented
> somewhere. Unfortunately, v4l framework does not seem to provide good
> place where to put autofocus. I believe, long-term, we'll need some
> kind of "video server" providing this kind of services.
> 
> Anyway, we probably don't want autofocus in kernel (even through some
> cameras do it in hardware), and we probably don't want autofocus in
> each and every user application.
> 
> So what remains is libv4l2. 

IMO, the best place for autofocus is at libv4l2. Putting it on a
separate "video server" application looks really weird for me.

Btw, libv4l2 already has some autotools for auto gain and auto
white balance. See the implementation under:
	lib/libv4lconvert/processing

The libv4l internal controls can be seen at:
	lib/libv4lconvert/control/libv4lcontrol.h

The ones implemented by the processing part of the library are:

$ git grep v4lcontrol_get_ctrl lib/libv4lconvert/processing/
lib/libv4lconvert/processing/autogain.c:        autogain = v4lcontrol_get_ctrl(data->control, V4LCONTROL_AUTOGAIN);
lib/libv4lconvert/processing/autogain.c:        target = v4lcontrol_get_ctrl(data->control, V4LCONTROL_AUTOGAIN_TARGET);
lib/libv4lconvert/processing/gamma.c:   int gamma = v4lcontrol_get_ctrl(data->control, V4LCONTROL_GAMMA);
lib/libv4lconvert/processing/gamma.c:   gamma = v4lcontrol_get_ctrl(data->control, V4LCONTROL_GAMMA);
lib/libv4lconvert/processing/whitebalance.c:    wb = v4lcontrol_get_ctrl(data->control, V4LCONTROL_WHITEBALANCE);

I guess it shouldn't be hard to add an extra processing module
there for auto-focus.

> Now, this is in no way clean or complete,
> and functionality provided by sdl.c and asciicam.c probably _should_
> be in application, but... I'd like to get the code out there.
> 
> Oh and yes, I've canibalized decode_tm6000.c application instead of
> introducing my own. Autotools scare me, sorry.

Why replace decode_tm6000.c by something else? If you want to add another
test application, just place it on a new file.

I added a few notes together with the code, pointing the main things
I think it require changes, in order for me to do a better review
at the code. I didn't test nor tried to check the algorithms inside,
as the code, on its current state, requires rework and code cleanup.

> 
> Regards,
> 							Pavel
> 
> diff --git a/lib/libv4l2/asciicam.c b/lib/libv4l2/asciicam.c
> new file mode 100644
> index 0000000..5388967
> --- /dev/null
> +++ b/lib/libv4l2/asciicam.c
> @@ -0,0 +1,63 @@
> +/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
> +   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
> +
> +gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
> + */
> +#include <stdio.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +
> +#include <linux/videodev2.h>
> +
> +#define SIZE 10*1024*1024
> +
> +char buf[SIZE];
> +
> +void main(void)

Please don't add a new application under lib/. It is fine if you want
some testing application, if the ones there aren't enough, but please
place it under contrib/test/.

You should likely take a look at v4l2grab first, as it could have
almost everything you would need.

> +{
> +  int fd = v4l2_open("/dev/video2", O_RDWR);

Hardcoding /dev/video2 doesn't seem a good idea.

> +  int i;
> +  static struct v4l2_format fmt;
> +
> +#if 1
> +  fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +  fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
> +  fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
> +  fmt.fmt.pix.width = 640;
> +  fmt.fmt.pix.height = 480;
> +  if (fmt.fmt.pix.pixelformat != 'RGB3') /* v4l2_fourcc('R', 'G', 'B', '3'); */
> +    printf("hmm. strange format?\n");
> +  
> +  printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
> +#endif
> +
> +  for (i=0; i<500; i++) {
> +    int num = v4l2_read(fd, buf, SIZE);
> +    int y,x;
> +    
> +    printf("%d\n", num);
> +#if 0
> +    for (y = 0; y < 25; y++) {
> +      for (x = 0; x < 80; x++) {
> +	int y1 = y * 480/25;
> +	int x1 = x * 640/80;
> +	int c = buf[fmt.fmt.pix.width*3*y1 + 3*x1] +
> +	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 1] +
> +	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 2];
> +
> +	if (c < 30) c = ' ';
> +	else if (c < 60) c = '.';
> +	else if (c < 120) c = '_';
> +	else if (c < 180) c = 'o';
> +	else if (c < 300) c = 'x';
> +	else if (c < 400) c = 'X';
> +	else c = '#';
> +	  
> +	printf("%c", c);
> +      }
> +      printf("\n");
> +    }

IMHO, it would be better to use aalib. Btw, xawtv3 has a code example
using it, under:
	console/ttv.c

As it already uses libv4l, prhaps you could use it, instead of adding
a new ascii app.

> +#endif    
> +  }
> +}
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..af740a7 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */
>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -70,6 +71,14 @@
>  	} while (0)
>  
>  #define MIN(a, b) (((a) < (b)) ? (a) : (b))
> +#define V4L2_MAX_SUBDEVS 16
> +
> +#define V4L2_MAX_FOCUS 10
> +struct v4l2_focus_step {
> +	int num;
> +	int sharpness[V4L2_MAX_FOCUS];
> +	int position[V4L2_MAX_FOCUS];
> +};
>  
>  struct v4l2_dev_info {
>  	int fd;
> @@ -104,6 +113,14 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
> +  	int exposure; 
> +  	int frame;
> +  	int focus;
> +
> +	/* Autofocus parameters */
> +	int focus_frame, focus_exposure, focus_step, sh_prev;
> +	struct v4l2_focus_step focus_step_params;
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..3b84437 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */

Don't add editor-specific macros inside the code. Not everybody uses emacs.

>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -100,6 +101,13 @@ static struct v4l2_dev_info devices[V4L2_MAX_DEVICES] = {
>  };
>  static int devices_used;
>  
> +#include "sdl.c"
> +
> +static struct sdl sdl;
> +
> +int v4l2_get_index(int fd);
> +void my_main(void);
> +

The above looks really odd. Why do you want to make libv4l2 dependent
on sdl?

>  static int v4l2_ensure_convert_mmap_buf(int index)
>  {
>  	if (devices[index].convert_mmap_buf != MAP_FAILED) {
> @@ -311,7 +319,409 @@ static int v4l2_queue_read_buffer(int index, int buffer_index)
>  	return 0;
>  }
>  
> -static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
> +static void v4l2_paint(char *buf, int x1, int y1, const struct v4l2_format *fmt)
> +{
> +  int y, x;
> +  int width = fmt->fmt.pix.width;
> +  printf("width = %d\n", width);
> +    
> +    for (y = 0; y < 25; y++) {
> +      for (x = 0; x < 80; x++) {
> +	int c = buf[width*4*y1 + 4*x1] +
> +	  buf[width*4*y1 + 4*x1 + 1] +
> +	  buf[width*4*y1 + 4*x1 + 2];
> +
> +	if (c < 30) c = ' ';
> +	else if (c < 60) c = '.';
> +	else if (c < 120) c = '_';
> +	else if (c < 180) c = 'o';
> +	else if (c < 300) c = 'x';
> +	else if (c < 400) c = 'X';
> +	else c = '#';
> +	  
> +	printf("%c", c);
> +      }
> +      printf("\n");
> +    }
> +}

Please don't add anything like that inside the library.

> +
> +#define SX 80
> +#define SY 25
> +#define BUCKETS 20
> +
> +static void v4l2_histogram(unsigned char *buf, int cdf[], struct v4l2_format *fmt)
> +{
> +    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
> +      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
> +	pixel p = buf_pixel(fmt, buf, x, y);
> +	
> +	int b;
> +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> +	b = p.r + 2*p.g + p.b;
> +	b = (b * BUCKETS)/(256);
> +	cdf[b]++;
> +      }
> +}
> +
> +static long v4l2_sharpness(unsigned char *buf, struct v4l2_format *fmt)
> +{
> +  int h = fmt->fmt.pix.height;
> +  int w = fmt->fmt.pix.width;
> +  long r = 0;
> +
> +    for (int y = h/3; y < h-h/3; y+=h/9)
> +      for (int x = w/3; x < w-w/3; x++) {
> +	pixel p1 = buf_pixel(fmt, buf, x, y);
> +	pixel p2 = buf_pixel(fmt, buf, x+2, y);
> +	
> +	int b1, b2;
> +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> +	b1 = p1.r + 2*p1.g + p1.b;
> +	b2 = p2.r + 2*p2.g + p2.b;
> +
> +	int v;
> +	v = (b1-b2)*(b1-b2);
> +	if (v > 36)
> +		r+=v;
> +      }
> +
> +    return r;
> +}

IMO, the above belongs to a separate processing module under
	lib/libv4lconvert/processing/

> +
> +int v4l2_set_exposure(int fd, int exposure)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +
> +	struct v4l2_control ctrl;
> +	ctrl.id = V4L2_CID_EXPOSURE;
> +	ctrl.value = exposure;
> +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> +	  printf("Could not set exposure\n");
> +	}
> +	return 0;
> +}

Shouldn't it be together with lib/libv4lconvert/processing/autogain.c,
perhaps as an alternative implementation, if what's there is not enough?

> +
> +int v4l2_set_gain(int fd, int gain)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +	
> +	struct v4l2_control ctrl;
> +	ctrl.id = 0x00980913;

Don't hardcode control numbers here.

> +	ctrl.value = gain;
> +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> +	  printf("Could not set exposure\n");
> +	}
> +	return 0;
> +}
> +
> +int v4l2_set_focus(int fd, int diopt)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_focus called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +
> +	struct v4l2_control ctrl;
> +	ctrl.id = V4L2_CID_FOCUS_ABSOLUTE;
> +	ctrl.value = diopt;
> +	if (ioctl(devices[index].subdev_fds[1], VIDIOC_S_CTRL, &ctrl) < 0) {
> +		printf("Could not set focus\n");
> +	}
> +	return 0;
> +}
> +
> +#define LESS 1
> +#define MID 0
> +#define MORE 2
> +
> +static void v4l2_start_focus_step(struct v4l2_focus_step *fs, int focus, int step)
> +{
> +	int i;
> +
> +	fs->num = 3;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->sharpness[i] = -1;
> +		fs->position[i] = -1;
> +	}
> +
> +	fs->position[LESS] = focus - step;
> +	fs->position[MID] = focus;
> +	fs->position[MORE] = focus + step;
> +}
> +
> +static int v4l2_focus_get_best(struct v4l2_focus_step *fs)
> +{
> +	int i, max = -1, maxi = -1;
> +	
> +	for (i=0; i<fs->num; i++)
> +		if (max < fs->sharpness[i]) {
> +			max = fs->sharpness[i];
> +			maxi = i;
> +		}
> +
> +	return maxi;
> +}
> +
> +static void v4l2_auto_focus_continuous(struct v4l2_dev_info *m, int sh)
> +{
> +	int f = m->frame - m->focus_frame;
> +	int step;
> +	const int max_step = 300, min_step = 20;
> +	struct v4l2_focus_step *fs = &m->focus_step_params;
> +
> +	if (m->focus_step == 0 || m->focus_step > max_step) {
> +		printf("step reset -- max\n");
> +		m->focus_step = max_step;
> +	}
> +	if (m->focus_step < min_step) {
> +		printf("step reset -- 10 (%d)\n", m->focus_step);
> +		m->focus_step = min_step;
> +		/* It takes cca 5.7 seconds to achieve the focus:
> +		   0.76user 0.30system 5.66 (0m5.661s) elapsed 18.72%CPU
> +		 */

The above note seems hardware dependent. 

> +		printf("Focused at %d\n", m->focus);
> +		exit(0);
> +	}
> +
> +	step = m->focus_step;
> +
> +	if (m->exposure != m->focus_exposure) {
> +		m->focus_frame = m->frame;
> +		m->focus_exposure = m->exposure;
> +		v4l2_start_focus_step(fs, m->focus, m->focus_step);
> +		return;
> +	}
> +	if (m->focus < step) {
> +		m->focus = step;
> +	}
> +
> +	const int every = 3;
> +	if (f%every)
> +		return;
> +
> +	{
> +		int i = f/every;
> +
> +		if (i == 0) {
> +			printf("Can not happen?\n");
> +			return;
> +		}
> +		i--;
> +		if (i < fs->num)
> +			v4l2_set_focus(m->fd, fs->position[i]);
> +		if (i > 0)
> +			fs->sharpness[i-1] = sh;
> +		if (i < fs->num)
> +			return;
> +	}
> +	int i;
> +	for (i=0; i<fs->num; i++) {
> +		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
> +	}

You should probably print something only if log is enabled. Take a look
at lib/libv4l2/log.c. Same applies to similar printf() calls.

> +	int best = v4l2_focus_get_best(fs);
> +	if ((fs->sharpness[best] < m->sh_prev) && (m->focus_step < max_step)) {
> +		m->focus_step *=2;
> +		m->sh_prev = m->sh_prev * 0.9;
> +		printf("step up %d\n", m->focus_step);
> +	} else if (best == LESS) {
> +		printf("less ");
> +		m->focus -= step;
> +	} else if (best == MORE) {
> +		printf("more ");
> +		m->focus += step;
> +	} else {
> +		m->sh_prev = fs->sharpness[MID];
> +		m->focus_step = m->focus_step / 2;
> +		printf("step %d ", m->focus_step);
> +	}
> +	m->focus_frame = m->frame;
> +	v4l2_start_focus_step(fs, m->focus, m->focus_step);
> +	printf("Focus now %d\n", m->focus);
> +}
> +
> +static void v4l2_start_focus_sweep(struct v4l2_focus_step *fs, int focus, int step)
> +{
> +	int i;
> +
> +	fs->num = V4L2_MAX_FOCUS;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->sharpness[i] = -1;
> +		fs->position[i] = -1;
> +	}
> +
> +	int f = focus;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->position[i] = f;
> +		f += step;
> +	}
> +}
> +
> +static void v4l2_auto_focus_single(struct v4l2_dev_info *m, int sh)
> +{
> +	int f = m->frame - m->focus_frame;
> +	int step;
> +	struct v4l2_focus_step *fs = &m->focus_step_params;
> +
> +	if (m->focus_step == 0) {
> +		printf("step reset -- max\n");
> +		m->focus_step = 1;
> +	}
> +
> +	if (m->exposure != m->focus_exposure) {
> +		m->focus_frame = m->frame;
> +		m->focus_exposure = m->exposure;
> +		v4l2_start_focus_sweep(fs, 0, 100);
> +		return;
> +	}
> +
> +	const int every = 3;
> +	if (f%every)
> +		return;
> +
> +	{
> +		int i = f/every;
> +
> +		if (i == 0) {
> +			printf("Can not happen?\n");
> +			return;
> +		}
> +		i--;
> +		if (i < fs->num)
> +			v4l2_set_focus(m->fd, fs->position[i]);
> +		if (i > 0)
> +			fs->sharpness[i-1] = sh;
> +		if (i < fs->num)
> +			return;
> +	}
> +#if 0
> +	int i;
> +	for (i=0; i<fs->num; i++) {
> +		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
> +	}
> +#endif
> +	int best = v4l2_focus_get_best(fs);
> +	m->focus_frame = m->frame;
> +	switch (m->focus_step) {
> +	case 1:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		v4l2_start_focus_sweep(fs, fs->position[best] - 50, 10);
> +		m->focus_step = 2;
> +		break;
> +	case 2:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		v4l2_start_focus_sweep(fs, fs->position[best] - 10, 2);
> +		m->focus_step = 3;
> +		break;
> +	case 3:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		printf("done.\n");
> +		exit(0);
> +	}
> +}
> +
> +
> +static void v4l2_auto_exposure(int index, struct v4l2_buffer *buf)
> +{
> +	struct v4l2_format *fmt;
> +	int cdf[BUCKETS] = { 0, };
> +	int i;
> +
> +	fmt = &devices[index].src_fmt;
> +
> +	v4l2_histogram(devices[index].frame_pointers[buf->index], cdf, fmt);
> +
> +#if 0
> +	printf("hist: ");
> +	for (i = 0; i<BUCKETS; i++)
> +		printf("%d ", cdf[i]);
> +	printf("\n");
> +#endif
> +	for (i=1; i<BUCKETS; i++)
> +		cdf[i] += cdf[i-1];
> +
> +	int b = BUCKETS;
> +	int brightPixels = cdf[b-1] - cdf[b-8];
> +	int targetBrightPixels = cdf[b-1]/50;
> +	int maxSaturatedPixels = cdf[b-1]/200;
> +	int saturatedPixels = cdf[b-1] - cdf[b-2];
> +	// how much should I change brightness by
> +	float adjustment = 1.0f;
> +#if 0
> +	printf( "AutoExposure: totalPixels: %d,"
> +		"brightPixels: %d, targetBrightPixels: %d,"
> +		"saturatedPixels: %d, maxSaturatedPixels: %d\n",
> +		cdf[b-1], brightPixels, targetBrightPixels,
> +		saturatedPixels, maxSaturatedPixels);
> +#endif
> +	  
> +	if (saturatedPixels > maxSaturatedPixels) {
> +		// first don't let things saturate too much
> +		adjustment = 1.0f - ((float)(saturatedPixels - maxSaturatedPixels))/cdf[b-1];
> +	} else if (brightPixels < (targetBrightPixels - (saturatedPixels * 4))) {
> +		// increase brightness to try and hit the desired number of well exposed pixels
> +		int l = b-6;
> +		while (brightPixels < targetBrightPixels && l > 0) {
> +			brightPixels += cdf[l];
> +			brightPixels -= cdf[l-1];
> +			l--;
> +		}
> +
> +		// that level is supposed to be at b-11;
> +		adjustment = ((float) (b-6+1))/(l+1);
> +	} else {
> +		// we're not oversaturated, and we have enough bright pixels. Do nothing.
> +	}
> +
> +	{
> +		float limit = 4;
> +		if (adjustment > limit) { adjustment = limit; }
> +		if (adjustment < 1/limit) { adjustment = 1/limit; }
> +	}
> +	  
> +	if (!devices[index].exposure)
> +		devices[index].exposure = 1;
> +	devices[index].exposure *= adjustment;
> +	if (adjustment != 1.)
> +		printf( "AutoExposure: adjustment: %f exposure %d\n", adjustment, devices[index].exposure);
> +
> +	v4l2_set_exposure(devices[index].fd, devices[index].exposure);
> +}
> +
> +static void v4l2_statistics(int index, struct v4l2_buffer *buf)
> +{
> +	unsigned char *b;
> +	struct v4l2_format *fmt;
> +
> +	fmt = &devices[index].src_fmt;
> +	b = devices[index].frame_pointers[buf->index];
> +
> +	if (!(devices[index].frame%3))
> +		v4l2_auto_exposure(index, buf);
> +	
> +	int sh = v4l2_sharpness(b, fmt);
> +	v4l2_auto_focus_single(&devices[index], sh);
> +	
> +	devices[index].frame++;
> +	if (!(devices[index].frame%4))
> +		sdl_render(&sdl, b, fmt);
> +}
> +
> +int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  		unsigned char *dest, int dest_size)
>  {
>  	const int max_tries = V4L2_IGNORE_FIRST_FRAME_ERRORS + 1;
> @@ -345,6 +755,13 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  			errno = -EINVAL;
>  			return -1;
>  		}
> +		
> +#if 1
> +		v4l2_statistics(index, buf);
> +#endif
> +#if 0
> +		/* This is rather major eater of CPU time. CPU time goes from 80% to 4% 
> +		   when conversion is disabled. */
>  
>  		result = v4lconvert_convert(devices[index].convert,
>  				&devices[index].src_fmt, &devices[index].dest_fmt,
> @@ -352,7 +769,7 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  				buf->bytesused, dest ? dest : (devices[index].convert_mmap_buf +
>  					buf->index * devices[index].convert_mmap_frame_size),
>  				dest_size);
> -
> +#endif
>  		if (devices[index].first_frame) {
>  			/* Always treat convert errors as EAGAIN during the first few frames, as
>  			   some cams produce bad frames at the start of the stream
> @@ -789,18 +1206,24 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +
> +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> +	       devices[index].subdev_fds[1]);
> +
>  	V4L2_LOG("open: %d\n", fd);
>  
>  	return fd;
>  }
>  
>  /* Is this an fd for which we are emulating v4l1 ? */
> -static int v4l2_get_index(int fd)
> +int v4l2_get_index(int fd)
>  {
>  	int index;
>  
> @@ -823,6 +1246,10 @@ int v4l2_close(int fd)
>  {
>  	int index, result;
>  
> +	if (fd == -2) {
> +	  my_main();
> +	}
> +

That looks a hack!

>  	index = v4l2_get_index(fd);
>  	if (index == -1)
>  		return SYS_CLOSE(fd);
> @@ -1782,3 +2209,65 @@ int v4l2_get_control(int fd, int cid)
>  			(qctrl.maximum - qctrl.minimum) / 2) /
>  		(qctrl.maximum - qctrl.minimum);
>  }
> +
> +void v4l2_debug(void)
> +{
> +	printf("debug\n");
> +}
> +
> +/* ------------------------------------------------------------------ */
> +
> +#define SIZE 10*1024*1024
> +
> +char buf[SIZE];
> +
> +void my_main(void)
> +{
> +	int fd = v4l2_open("/dev/video2", O_RDWR);
> +	int i;
> +	static struct v4l2_format fmt;
> +
> +	fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +	fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
> +	fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
> +	fmt.fmt.pix.width = 640;
> +	fmt.fmt.pix.height = 480;
> +
> +	v4l2_set_gain(fd, 300);
> +	v4l2_set_exposure(fd, 10000);
> +	v4l2_set_focus(fd, 0);
> +
> +
> +	printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
> +
> +	printf("capture is %d, %d\n", fmt.fmt.pix.width, fmt.fmt.pix.height);
> +	/* factor == 2 still fits window, but very slow. factor == 6 .. */
> +	/* factor needs to be odd, otherwise ... fun with BA10 format. */
> +	sdl_init(&sdl, fmt.fmt.pix.width, fmt.fmt.pix.height, 5);
> +  
> +	v4l2_debug();
> +
> +	/* In 800x600 "raw" mode, this should take cca 17.8 seconds (without
> +	   sdl output. CPU usage should be cca 5% without conversion). That's 28 fps.
> +	   (benchmark with i<500)
> +	*/
> +	for (i=0; i<50000; i++) {
> +		int num = v4l2_read(fd, buf, SIZE);
> +
> +		if (i==490) {
> +			printf("Focus to closest.... -------------------\n");
> +			v4l2_set_focus(fd, 99999);
> +		}
> +    
> +#if 0
> +		v4l2_paint(buf, 640/80, 480/25, &fmt);
> +#endif
> +		/* Over USB connection, rendering every single frame slows
> +		   execution down from 23 seconds to 36 seconds. */
> +#if 0
> +		if (!(i%4))
> +			sdl_render(&sdl, buf, &fmt);
> +#endif
> +	}
> +  
> +}
> diff --git a/lib/libv4l2/sdl.c b/lib/libv4l2/sdl.c
> new file mode 100644
> index 0000000..17a8d24
> --- /dev/null
> +++ b/lib/libv4l2/sdl.c
> @@ -0,0 +1,530 @@
> +/* -*- c-file-style: "linux" -*- */
> +/* SDL support.
> +
> +   Copyright 2017 Pavel Machek, LGPL
> +*/
> +
> +#include <SDL2/SDL.h>
> +#include <SDL2/SDL_image.h>

If you're adding a SDL-specific application, you'll need to add the 
needed autoconf bits to detect if SDL devel package is installed,
auto-disabling it if not.

Yet, I don't think that SDL should be part of the library, but,
instead, part of some application.

> +
> +struct sdl {
> +	SDL_Window *window;
> +	SDL_Surface *liveview, *screen;
> +
> +	int wx, wy;
> +	int sx, sy;
> +	int bx, by;
> +	int factor;
> +	float focus, gain, exposure, do_focus, do_exposure; 
> +};
> +
> +#if 0
> +void loop(void) {
> +	int done;
> +	SDL_Event event;
> +
> +	while(!done){ //While program isn't done                                   
> +		while(SDL_PollEvent(&event)){ //Poll events                        
> +			switch(event.type){ //Check event type                     
> +			case SDL_QUIT: //User hit the X (or equivelent)            
> +				done = true; //Make the loop end                   
> +				break; //We handled the event                      
> +			} //Finished with current event                            
> +		} //Done with all events for now                                   
> +	} //Program done, exited                                                   
> +}
> +#endif
> +
> +typedef struct {
> +	Uint8 r;
> +	Uint8 g;
> +	Uint8 b;
> +	Uint8 alpha;
> +} pixel;
> +
> +#define d_raw 1
> +
> +void sfc_put_pixel(SDL_Surface* liveview, int x, int y, pixel *p)
> +{
> +	Uint32* p_liveview = (Uint32*)liveview->pixels;
> +	p_liveview += y*liveview->w+x;
> +	*p_liveview = SDL_MapRGBA(liveview->format,p->r,p->g,p->b,p->alpha);
> +}
> +
> +#if 0
> +int pix_exposure(float x)
> +{
> +	return sy*x / 199410.0;
> +}
> +
> +int pix_gain(float x)
> +{
> +	return sy*x / 16.0;
> +}
> +
> +int render_statistics(SDL_Surface* liveview)
> +{
> +	pixel white;
> +	white.r = (Uint8)0xff;
> +	white.g = (Uint8)0xff;
> +	white.b = (Uint8)0xff;
> +	white.alpha = (Uint8)128;
> +
> +	//printf("Stats: focus %d, gain %d, exposure %d\n", focus, gain, exposure);
> +
> +	for (int x=0; x<sx && x<sx*focus; x++)
> +		put_pixel(liveview, x, 0, &white);
> +
> +	for (int y=0; y<sy && y<pix_gain(gain); y++)
> +		put_pixel(liveview, 0, y, &white);
> +
> +	for (int y=0; y<sy && y<pix_exposure(exposure); y++)
> +		put_pixel(liveview, sx-1, y, &white);
> +
> +	for (int x=0; x<sx; x++)
> +		put_pixel(liveview, x, sy-1, &white);
> +
> +	return 0;
> +}
> +#endif
> +
> +void sdl_begin_paint(struct sdl *m) {
> +	//Fill the surface white                                                   
> +	SDL_FillRect(m->liveview, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
> +
> +	SDL_LockSurface(m->liveview);
> +}
> +
> +void sdl_finish_paint(struct sdl *m) {
> +	SDL_UnlockSurface(m->liveview);
> +	SDL_Rect rcDest = { m->bx, m->by, m->sx, m->sy };
> +
> +	SDL_BlitSurface(m->liveview, NULL, m->screen, &rcDest);
> +	//Update the surface                                                       
> +	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
> +}
> +  
> +void sdl_paint_image(struct sdl *m, char **xpm, int x, int y) {
> +	SDL_Surface *image = IMG_ReadXPMFromArray(xpm);
> +	if (!image) {
> +		printf("IMG_Load: %s\n", IMG_GetError());
> +		exit(1);
> +	}
> +
> +	int x_pos = x - image->w/2, y_pos = y - image->h/2;
> +
> +	SDL_Rect rcDest = { x_pos, y_pos, image->w, image->h };
> +	int r = SDL_BlitSurface ( image, NULL, m->screen, &rcDest );
> +
> +	if (r) {
> +		printf("Error blitting: %s\n", SDL_GetError());
> +		exit(1);
> +	}
> +	SDL_FreeSurface ( image );
> +}
> +
> +void sdl_paint_ui(struct sdl *m) {
> +	static char *wait_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"....########....",
> +		".....#....#.....",
> +		".....#....#.....",
> +		"......#..#......",
> +		".......##.......",
> +		"......#..#......",
> +		".....#....#.....",
> +		".....#....#.....",
> +		"....########....",
> +	};
> +
> +	static char *ok_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"...............#",
> +		"............###.",
> +		"..........##....",
> +		"#.......##......",
> +		".#.....#........",
> +		"..#...#.........",
> +		"..#..#..........",
> +		"...##...........",
> +		"...#............",
> +	};
> +
> +	static char *exit_xpm[] = {
> +		"16 9 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"....x......x....",
> +		".....x....x.....",
> +		"......x..x......",
> +		".......xx.......",
> +		".......xx.......",
> +		"......x..x......",
> +		".....x....x.....",
> +		"....x......x....",
> +		"................",
> +	};
> +
> +	static char *f1m_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"....##..........",
> +		"...#.#..........",
> +		"..#..#..........",
> +		".....#...#.#.##.",
> +		".....#...##.#..#",
> +		".....#...#..#..#",
> +		".....#...#..#..#",
> +		".....#...#..#..#",
> +		"................",
> +	};
> +
> +	static char *f25cm_xpm[] = {
> +		"16 9 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".xxx..xxxx......",
> +		"x...x.x.........",
> +		"...x..xxx.......",
> +		"..x......x..xx.x",
> +		".x.......x.x.xxx",
> +		"xxxxx.xxx...xxxx",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	static char *iso400_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"x..x.xxxx.xxxx..",
> +		"x..x.x..x.x..x..",
> +		"xxxx.x..x.x..x..",
> +		"...x.x..x.x..x..",
> +		"...x.xxxx.xxxx..",
> +		"................",
> +		".x..xx..x.......",
> +		".x.x...x.x......",
> +		".x..x..x.x......",
> +		".x...x.x.x......",
> +		".x.xx...x.......",
> +		"................",
> +	};
> +
> +	static char *time_1_100_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".x....x.........",
> +		".x...x..........",
> +		".x..x...........",
> +		".x.x............",
> +		"................",
> +		"..x.xxxx.xxxx...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.xxxx.xxxx...",
> +		"................",
> +	};
> +
> +	static char *time_1_10_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".x....x.........",
> +		".x...x..........",
> +		".x..x...........",
> +		".x.x............",
> +		"................",
> +		"..x.xxxx........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.xxxx........",
> +		"................",
> +	};
> +
> +	static char *af_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".....xxxxxxx....",
> +		".....x..........",
> +		".....x..........",
> +		".x...xxxxx......",
> +		"x.x..x..........",
> +		"xxx..x..........",
> +		"x.x..x..........",
> +		"x.x..x..........",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	static char *ae_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".....xxxxxxx....",
> +		".....x..........",
> +		".....x..........",
> +		".x...xxxxx......",
> +		"x.x..x..........",
> +		"xxx..x..........",
> +		"x.x..x..........",
> +		"x.x..xxxxxxx....",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +    
> +	static char *not_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"......xxxxx.....",
> +		"....xx.....xx...",
> +		"...x.........x..",
> +		"..x........xx.x.",
> +		"..x......xx...x.",
> +		".x.....xx......x",
> +		".x...xx........x",
> +		"..xxx.........x.",
> +		"..x...........x.",
> +		"...x.........x..",
> +		"....xx.....xx...",
> +		"......xxxxx.....",
> +	};
> +
> +	static char *empty_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	SDL_FillRect(m->screen, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
> +	sdl_paint_image(m, wait_xpm,  m->wx/2,     m->wy/2);
> +	sdl_paint_image(m, ok_xpm,    m->wx-m->bx/2,  m->wy-m->by/2);
> +	sdl_paint_image(m, exit_xpm,  m->bx/2,     m->wy-m->by/2);
> +	sdl_paint_image(m, f1m_xpm,   m->bx+m->sx/20, m->by/2);
> +	sdl_paint_image(m, f25cm_xpm, m->bx+m->sx/5,  m->by/2);
> +
> +	sdl_paint_image(m, af_xpm,    m->bx/2,     m->by/2);
> +	if (!m->do_focus) {
> +		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
> +	}
> +	sdl_paint_image(m, ae_xpm,    m->wx-m->bx/2,  m->by/2);
> +	if (!m->do_exposure) {
> +		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
> +	}
> +
> +#if 0
> +	sdl_paint_image(m, time_1_100_xpm, m->wx-m->bx/2, m->by+pix_exposure(10000));
> +	sdl_paint_image(m, time_1_10_xpm,  m->wx-m->bx/2, m->by+pix_exposure(100000));
> +	sdl_paint_image(m, iso400_xpm,     m->bx/2,    m->by+pix_gain(4.0));
> +#endif
> +		
> +	SDL_UpdateWindowSurface(m->window);
> +}
> +
> +void fmt_print(struct v4l2_format *fmt)
> +{
> +	int f;
> +	printf("Format: %dx%d. ", fmt->fmt.pix.width, fmt->fmt.pix.height);
> +	printf("%x ", fmt->fmt.pix.pixelformat);
> +	f = fmt->fmt.pix.pixelformat;
> +	for (int i=0; i<4; i++) {
> +		printf("%c", f & 0xff);
> +		f >>= 8;
> +	}
> +	printf("\n");
> +}
> +
> +pixel buf_pixel(struct v4l2_format *fmt, unsigned char *buf, int x, int y)
> +{
> +	pixel p = { 0, 0, 0, 0 };
> +	int pos = x + y*fmt->fmt.pix.width;
> +	int b;
> +
> +	p.alpha = 128;
> +
> +	switch (fmt->fmt.pix.pixelformat) {
> +	case '01AB': /* BA10, aka GRBG10, 
> +			https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-srggb10.html?highlight=ba10
> +		     */
> +		b = buf[pos*2];
> +		b += buf[pos*2+1] << 8;
> +		b /= 4;
> +		if ((y % 2) == 0) {
> +			switch (x % 2) {
> +			case 0: p.g = b/2; break;
> +			case 1: p.r = b; break;
> +			}
> +		} else {
> +			switch (x % 2) {
> +			case 0: p.b = b; break;
> +			case 1: p.g = b/2; break;
> +			}
> +		}
> +		break;
> +
> +	case V4L2_PIX_FMT_RGB24:
> +		pos *= 4;
> +		p.r = buf[pos];
> +		p.g = buf[pos+1];
> +		p.b = buf[pos+2];
> +		break;
> +
> +	default:
> +		printf("Wrong pixel format!\n");
> +		fmt_print(fmt);
> +		exit(1);
> +	}
> +
> +	return p;
> +}
> +
> +void sdl_render(struct sdl *m, unsigned char *buf, struct v4l2_format *fmt)
> +{
> +	if (!m->window) 
> +		return;
> +	sdl_begin_paint(m);    
> +	/* do your rendering here */
> +
> +	for (int y = 0; y < m->sy; y++)
> +		for (int x = 0; x < m->sx; x++) {
> +			pixel p = buf_pixel(fmt, buf, x*m->factor, y*m->factor);
> +			p.alpha = 128;
> +			sfc_put_pixel(m->liveview, x, y, &p);
> +		}
> +
> +	//render_statistics(liveview);
> +
> +	sdl_finish_paint(m);
> +}
> +
> +#if 0
> +void imageStatistics(FCam::Image img)
> +{
> +	int sx, sy;
> +
> +	sx = img.size().width;                                                          
> +	sy = img.size().height;                                                         
> +	printf("Image is %d x %d, ", sx, sy);                                           
> +
> +	printf("(%d) ", img.brightness(sx/2, sy/2));
> +
> +	int dark = 0;                                                                   
> +	int bright = 0;                                                                 
> +	int total = 0;                                                                  
> +#define RATIO 10
> +	for (int y = sy/(2*RATIO); y < sy; y += sy/RATIO)                               
> +		for (int x = sx/(2*RATIO); x < sx; x += sx/RATIO) {                    
> +			int br = img.brightness(x, y);                               
> +
> +			if (!d_raw) {
> +				/* It seems real range is 60 to 218 */
> +				if (br > 200)
> +					bright++;                                               
> +				if (br < 80)
> +					dark++;
> +			} else {
> +				/* there's a lot of noise, it seems black is commonly 65..71,
> +				   bright is cca 1023 */
> +				if (br > 1000)
> +					bright++;
> +				if (br < 75)
> +					dark++;
> +			}
> +			total++;                                                      
> +		}
> +
> +	printf("%d dark %d bri,", dark, bright);                     
> +
> +	long sharp = 0;
> +	for (int y = sy/3; y < 2*(sy/3); y+=sy/12) {
> +		int b = -1;
> +		for (int x = sx/3; x < 2*(sx/3); x++) {
> +			int b2;                                                                
> +			b2 = img.brightness(x, y/2);                                          
> +			if (b != -1)
> +				sharp += (b-b2) * (b-b2);                                              
> +			b = b2;                                                                
> +		}
> +	}
> +	printf("sh %d\n", sharp);
> +}
> +#endif
> +
> +
> +void sdl_init(struct sdl *m, int _sx, int _sy, int _factor)
> +{
> +	printf("Initing SDL\n");
> +
> +	if (SDL_Init(SDL_INIT_VIDEO) < 0) {
> +		printf("Could not init SDL\n");
> +		exit(1);
> +	}
> +
> +	atexit(SDL_Quit);
> +
> +	m->wx = 800;
> +	m->wy = 400;
> +
> +	m->window = SDL_CreateWindow( "Camera", SDL_WINDOWPOS_UNDEFINED,
> +				      SDL_WINDOWPOS_UNDEFINED, m->wx, m->wy,
> +				      SDL_WINDOW_SHOWN );
> +	if (m->window == NULL) {
> +		printf( "Window could not be created! SDL_Error: %s\n", SDL_GetError() );
> +	}
> +
> +	m->screen = SDL_GetWindowSurface(m->window);
> +	if (!m->screen) {
> +		printf("Couldn't create liveview\n");
> +		exit(1);
> +	}
> +
> +	m->sx = _sx;
> +	m->sy = _sy;
> +	m->factor = _factor;
> +
> +	m->sx /= m->factor;
> +	m->sy /= m->factor;
> +    
> +	m->focus = 0;
> +	m->gain = 0;
> +	m->exposure = 0;
> +
> +	m->bx = (m->wx-m->sx)/2;
> +	m->by = (m->wy-m->sy)/2;
> +
> +	m->liveview = SDL_CreateRGBSurface(0,m->sx,m->sy,32,0,0,0,0);
> +	if (!m->liveview) {
> +		printf("Couldn't create liveview\n");
> +		exit(1);
> +	}
> +	sdl_paint_ui(m);
> +}
> diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
> index d3d8936..7521ec8 100644
> --- a/lib/libv4lconvert/libv4lconvert.c
> +++ b/lib/libv4lconvert/libv4lconvert.c
> @@ -1205,6 +1205,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
>  			v4lconvert_swap_uv(src, dest, fmt);
>  			break;
>  		}
> +		default:
> +		  /* This is bad, fixme? */
>  		break;
>  
>  	case V4L2_PIX_FMT_YVU420:
> @@ -1228,6 +1230,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
>  		case V4L2_PIX_FMT_YVU420:
>  			memcpy(dest, src, width * height * 3 / 2);
>  			break;
> +		default:
> +		  /* This is bad, fixme? */
>  		}
>  		break;
>  
> diff --git a/utils/decode_tm6000/Makefile.am b/utils/decode_tm6000/Makefile.am
> index ac4e85e..f059e3c 100644
> --- a/utils/decode_tm6000/Makefile.am
> +++ b/utils/decode_tm6000/Makefile.am
> @@ -1,4 +1,4 @@
>  bin_PROGRAMS = decode_tm6000
>  decode_tm6000_SOURCES = decode_tm6000.c
> -decode_tm6000_LDADD = ../libv4l2util/libv4l2util.la
> +decode_tm6000_LDADD = ../libv4l2/libv4l2.la
>  decode_tm6000_LDFLAGS = $(ARGP_LIBS)
> diff --git a/utils/decode_tm6000/decode_tm6000.c b/utils/decode_tm6000/decode_tm6000.c
> index 4bffbdd..fda7e3b 100644
> --- a/utils/decode_tm6000/decode_tm6000.c
> +++ b/utils/decode_tm6000/decode_tm6000.c

Everything below it is completely wrong!

> @@ -1,365 +1,16 @@
> -/*
> -   decode_tm6000.c - decode multiplexed format from TM5600/TM6000 USB
> +/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
> +   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
>  
> -   Copyright (C) 2007 Mauro Carvalho Chehab <mchehab@infradead.org>
> -
> -   This program is free software; you can redistribute it and/or modify
> -   it under the terms of the GNU General Public License as published by
> -   the Free Software Foundation version 2.
> -
> -   This program is distributed in the hope that it will be useful,
> -   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> -   GNU General Public License for more details.
> -
> -   You should have received a copy of the GNU General Public License
> -   along with this program; if not, write to the Free Software
> -   Foundation, Inc., 51 Franklin Street, Suite 500, Boston, MA 02110-1335 USA.
> +gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
>   */
> -#include "../libv4l2util/v4l2_driver.h"
>  #include <stdio.h>
> -#include <string.h>
> -#include <argp.h>
> -#include <unistd.h>
>  #include <sys/types.h>
>  #include <sys/stat.h>
>  #include <fcntl.h>
> -#include <stdlib.h>
> -#include <errno.h>
> -
> -const char *argp_program_version="decode_tm6000 version 0.0.1";
> -const char *argp_program_bug_address="Mauro Carvalho Chehab <mchehab@infradead.org>";
> -const char doc[]="Decodes tm6000 proprietary format streams";
> -const struct argp_option options[] = {
> -	{"verbose",	'v',	0,	0,	"enables debug messages", 0},
> -	{"device",	'd',	"DEV",	0,	"uses device for reading", 0},
> -	{"output",	'o',	"FILE",	0,	"outputs raw stream to a file", 0},
> -	{"input",	'i',	"FILE",	0,	"parses a file, instead of a device", 0},
> -	{"freq",	'f',	"Freq",	0,	"station frequency, in MHz (default is 193.25)", 0},
> -	{"nbufs",	'n',	"quant",0,	"number of video buffers", 0},
> -	{"audio",	'a',	0,	0,	"outputs audio on stdout", 0},
> -	{"read",	'r',	0,	0,	"use read() instead of mmap method", 0},
> -	{ 0, 0, 0, 0, 0, 0 }
> -};
> -
> -static char outbuf[692224];
> -static int debug=0, audio=0, use_mmap=1, nbufs=4;
> -static float freq_mhz=193.25;
> -static char *devicename="/dev/video0";
> -static char *filename=NULL;
> -static enum {
> -	NORMAL,
> -	INPUT,
> -	OUTPUT
> -} mode = NORMAL;
> -
> -static FILE *fout;
> -
> -//const char args_doc[]="ARG1 ARG2";
> -
> -static error_t parse_opt (int key, char *arg, struct argp_state *state)
> -{
> -	switch (key) {
> -	case 'a':
> -		audio++;
> -		break;
> -	case 'r':
> -		use_mmap=0;
> -		break;
> -	case 'v':
> -		debug++;
> -		break;
> -	case 'd':
> -		devicename=arg;
> -		break;
> -	case 'i':
> -	case 'o':
> -		if (mode!=NORMAL) {
> -			argp_error(state,"You can't use input/output options simultaneously.\n");
> -			break;
> -		}
> -		if (key=='i')
> -			mode=INPUT;
> -		else
> -			mode=OUTPUT;
> -
> -		filename=arg;
> -		break;
> -	case 'f':
> -		freq_mhz=atof(arg);
> -		break;
> -	case 'n':
> -		nbufs=atoi(arg);
> -		if  (nbufs<2)
> -			nbufs=2;
> -		break;
> -	default:
> -		return ARGP_ERR_UNKNOWN;
> -	}
> -	return 0;
> -}
> -
> -static struct argp argp = {
> -	.options=options,
> -	.parser=parse_opt,
> -	.args_doc=NULL,
> -	.doc=doc,
> -};
>  
> -#define TM6000_URB_MSG_LEN 180
> -enum {
> -	TM6000_URB_MSG_VIDEO=1,
> -	TM6000_URB_MSG_AUDIO,
> -	TM6000_URB_MSG_VBI,
> -	TM6000_URB_MSG_PTS,
> -	TM6000_URB_MSG_ERR,
> -};
> -
> -const char *tm6000_msg_type[]= {
> -	"unknown(0)",	//0
> -	"video",	//1
> -	"audio",	//2
> -	"vbi",		//3
> -	"pts",		//4
> -	"err",		//5
> -	"unknown(6)",	//6
> -	"unknown(7)",	//7
> -};
> -
> -#define dprintf(fmt,arg...) \
> -	if (debug) fprintf(stderr, fmt, ##arg)
> -
> -static int recebe_buffer (struct v4l2_buffer *v4l2_buf, struct v4l2_t_buf *buf)
> -{
> -	dprintf("Received %zd bytes\n", buf->length);
> -fflush(stdout);
> -	memcpy (outbuf,buf->start,buf->length);
> -	return buf->length;
> -}
> -
> -
> -static int prepare_read (struct v4l2_driver *drv)
> -{
> -	struct v4l2_format fmt;
> -	double freq;
> -	int rc;
> -
> -	memset (drv,0,sizeof(*drv));
> -
> -	if (v4l2_open (devicename, 1,drv)<0) {
> -		perror ("Error opening dev");
> -		return -1;
> -	}
> -
> -	memset (&fmt,0,sizeof(fmt));
> -
> -	uint32_t pixelformat=V4L2_PIX_FMT_TM6000;
> -
> -	if (v4l2_gettryset_fmt_cap (drv,V4L2_SET,&fmt, 720, 480,
> -				    pixelformat,V4L2_FIELD_ANY)) {
> -		perror("set_input to tm6000 raw format");
> -		return -1;
> -	}
> -
> -	if (freq_mhz) {
> -		freq=freq_mhz * 1000 * 1000;
> -		rc=v4l2_getset_freq (drv,V4L2_SET, &freq);
> -		if (rc<0)
> -			printf ("Cannot set freq to %.3f MHz\n",freq_mhz);
> -	}
> -
> -	if (use_mmap) {
> -		printf("Preparing for receiving frames on %d buffers...\n",nbufs);
> -		fflush (stdout);
> -
> -		rc=v4l2_mmap_bufs(drv, nbufs);
> -		if (rc<0) {
> -			printf ("Cannot mmap %d buffers\n",nbufs);
> -			return -1;
> -		}
> -
> -//		v4l2_stop_streaming(&drv);
> -		rc=v4l2_start_streaming(drv);
> -		if (rc<0) {
> -			printf ("Cannot start streaming\n");
> -			return -1;
> -		}
> -	}
> -	printf("Waiting for frames...\n");
> -
> -	return 0;
> -}
> +#include <linux/videodev2.h>
>  
> -static int read_stream (struct v4l2_driver *drv, int fd)
> +void main(void)
>  {
> -	if (use_mmap) {
> -		fd_set fds;
> -		struct timeval tv;
> -		int r;
> -
> -		FD_ZERO (&fds);
> -		FD_SET (fd, &fds);
> -
> -		/* Timeout. */
> -		tv.tv_sec = 2;
> -		tv.tv_usec = 0;
> -
> -		r = select (fd + 1, &fds, NULL, NULL, &tv);
> -		if (-1 == r) {
> -			if (EINTR == errno) {
> -				perror ("select");
> -				return -errno;
> -			}
> -		}
> -
> -		if (0 == r) {
> -			fprintf (stderr, "select timeout\n");
> -			return -errno;
> -		}
> -
> -		return v4l2_rcvbuf(drv, recebe_buffer);
> -	} else {
> -		int size=read(fd, outbuf, sizeof(outbuf));
> -		return size;
> -	}
> -
> -	return 0;
> -}
> -
> -static int read_char (struct v4l2_driver *drv, int fd)
> -{
> -	static int sizebuf=0;
> -	static unsigned char *p=NULL;
> -	unsigned char c;
> -
> -	if (sizebuf<=0) {
> -		sizebuf=read_stream(drv,fd);
> -		if (sizebuf<=0)
> -			return -1;
> -		p=(unsigned char *)outbuf;
> -	}
> -	c=*p;
> -	p++;
> -	sizebuf--;
> -
> -	return c;
> -}
> -
> -
> -int main (int argc, char*argv[])
> -{
> -	int fd;
> -	unsigned int i;
> -	unsigned char buf[TM6000_URB_MSG_LEN], img[720*2*480];
> -	unsigned int  cmd, size, field, block, line, pos=0;
> -	unsigned long header=0;
> -	int           linesize=720*2,skip=0;
> -	struct v4l2_driver drv;
> -
> -	argp_parse (&argp, argc, argv, 0, 0, 0);
> -
> -	if (mode!=INPUT) {
> -		if (prepare_read (&drv)<0)
> -			return -1;
> -		fd=drv.fd;
> -	} else {
> -		/*mode == INPUT */
> -
> -		fd=open(filename,O_RDONLY);
> -		if (fd<0) {
> -			perror ("error opening a file for parsing");
> -			return -1;
> -		}
> -		dprintf("file %s opened for parsing\n",filename);
> -		use_mmap=0;
> -	}
> -
> -	if (mode==OUTPUT) {
> -		fout=fopen(filename,"w");
> -		if (!fout) {
> -			perror ("error opening a file to write");
> -			return -1;
> -		}
> -		dprintf("file %s opened for output\n",filename);
> -
> -		do {
> -			size=read_stream (&drv,fd);
> -
> -			if (size<=0) {
> -				close (fd);
> -				return -1;
> -			}
> -			dprintf("writing %d bytes\n",size);
> -			fwrite(outbuf,1, size,fout);
> -//			fflush(fout);
> -		} while (1);
> -	}
> -
> -
> -	while (1) {
> -		skip=0;
> -		header=0;
> -		do {
> -			int c;
> -			c=read_char (&drv,fd);
> -			if (c<0) {
> -				perror("read");
> -				return -1;
> -			}
> -
> -			header=(header>>8)&0xffffff;  
> -			header=header|(c<<24);
> -			skip++;
> -		} while ( (((header>>24)&0xff) != 0x47) );
> -
> -		/* split the header fields */
> -		size  = (((header & 0x7e)<<1) -1) * 4;
> -		block = (header>>7) & 0xf;
> -		field = (header>>11) & 0x1;
> -		line  = (header>>12) & 0x1ff;
> -		cmd   = (header>>21) & 0x7;
> -
> -		/* Read the remaining buffer */
> -		for (i=0;i<sizeof(buf);i++) {
> -			int c;
> -			c=read_char (&drv,fd);
> -			if (c<0) {
> -				perror("read");
> -				return -1;
> -			}
> -			buf[i]=c;
> -		}
> -
> -		/* FIXME: Mounts the image as field0+field1
> -			* It should, instead, check if the user selected
> -			* entrelaced or non-entrelaced mode
> -			*/
> -		pos=((line<<1)+field)*linesize+
> -					block*TM6000_URB_MSG_LEN;
> -
> -			/* Prints debug info */
> -		dprintf("0x%08x (skip %d), %s size=%d, line=%d, field=%d, block=%d\n",
> -				(unsigned int)header, skip,
> -				 tm6000_msg_type[cmd],
> -				 size, line, field, block);
> -
> -		/* Don't allow to write out of the buffer */
> -		if (pos+sizeof(buf) > sizeof(img))
> -			cmd = TM6000_URB_MSG_ERR;
> -
> -		/* handles each different URB message */
> -		switch(cmd) {
> -		case TM6000_URB_MSG_VIDEO:
> -			/* Fills video buffer */
> -			memcpy(buf,&img[pos],sizeof(buf));
> -		case TM6000_URB_MSG_AUDIO:
> -			if (audio)
> -				fwrite(buf,sizeof(buf),1,stdout);
> -//		case TM6000_URB_MSG_VBI:
> -//		case TM6000_URB_MSG_PTS:
> -		break;
> -		}
> -	}
> -	close(fd);
> -	return 0;
> +  v4l2_close(-2);
>  }
> 



Thanks,
Mauro

support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Hi Pavel,

Em Mon, 24 Apr 2017 11:30:59 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> For focus to be useful, we need autofocus implmented
> somewhere. Unfortunately, v4l framework does not seem to provide good
> place where to put autofocus. I believe, long-term, we'll need some
> kind of "video server" providing this kind of services.
> 
> Anyway, we probably don't want autofocus in kernel (even through some
> cameras do it in hardware), and we probably don't want autofocus in
> each and every user application.
> 
> So what remains is libv4l2. 

IMO, the best place for autofocus is at libv4l2. Putting it on a
separate "video server" application looks really weird for me.

Btw, libv4l2 already has some autotools for auto gain and auto
white balance. See the implementation under:
	lib/libv4lconvert/processing

The libv4l internal controls can be seen at:
	lib/libv4lconvert/control/libv4lcontrol.h

The ones implemented by the processing part of the library are:

$ git grep v4lcontrol_get_ctrl lib/libv4lconvert/processing/
lib/libv4lconvert/processing/autogain.c:        autogain = v4lcontrol_get_ctrl(data->control, V4LCONTROL_AUTOGAIN);
lib/libv4lconvert/processing/autogain.c:        target = v4lcontrol_get_ctrl(data->control, V4LCONTROL_AUTOGAIN_TARGET);
lib/libv4lconvert/processing/gamma.c:   int gamma = v4lcontrol_get_ctrl(data->control, V4LCONTROL_GAMMA);
lib/libv4lconvert/processing/gamma.c:   gamma = v4lcontrol_get_ctrl(data->control, V4LCONTROL_GAMMA);
lib/libv4lconvert/processing/whitebalance.c:    wb = v4lcontrol_get_ctrl(data->control, V4LCONTROL_WHITEBALANCE);

I guess it shouldn't be hard to add an extra processing module
there for auto-focus.

> Now, this is in no way clean or complete,
> and functionality provided by sdl.c and asciicam.c probably _should_
> be in application, but... I'd like to get the code out there.
> 
> Oh and yes, I've canibalized decode_tm6000.c application instead of
> introducing my own. Autotools scare me, sorry.

Why replace decode_tm6000.c by something else? If you want to add another
test application, just place it on a new file.

I added a few notes together with the code, pointing the main things
I think it require changes, in order for me to do a better review
at the code. I didn't test nor tried to check the algorithms inside,
as the code, on its current state, requires rework and code cleanup.

> 
> Regards,
> 							Pavel
> 
> diff --git a/lib/libv4l2/asciicam.c b/lib/libv4l2/asciicam.c
> new file mode 100644
> index 0000000..5388967
> --- /dev/null
> +++ b/lib/libv4l2/asciicam.c
> @@ -0,0 +1,63 @@
> +/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
> +   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
> +
> +gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
> + */
> +#include <stdio.h>
> +#include <sys/types.h>
> +#include <sys/stat.h>
> +#include <fcntl.h>
> +
> +#include <linux/videodev2.h>
> +
> +#define SIZE 10*1024*1024
> +
> +char buf[SIZE];
> +
> +void main(void)

Please don't add a new application under lib/. It is fine if you want
some testing application, if the ones there aren't enough, but please
place it under contrib/test/.

You should likely take a look at v4l2grab first, as it could have
almost everything you would need.

> +{
> +  int fd = v4l2_open("/dev/video2", O_RDWR);

Hardcoding /dev/video2 doesn't seem a good idea.

> +  int i;
> +  static struct v4l2_format fmt;
> +
> +#if 1
> +  fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +  fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
> +  fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
> +  fmt.fmt.pix.width = 640;
> +  fmt.fmt.pix.height = 480;
> +  if (fmt.fmt.pix.pixelformat != 'RGB3') /* v4l2_fourcc('R', 'G', 'B', '3'); */
> +    printf("hmm. strange format?\n");
> +  
> +  printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
> +#endif
> +
> +  for (i=0; i<500; i++) {
> +    int num = v4l2_read(fd, buf, SIZE);
> +    int y,x;
> +    
> +    printf("%d\n", num);
> +#if 0
> +    for (y = 0; y < 25; y++) {
> +      for (x = 0; x < 80; x++) {
> +	int y1 = y * 480/25;
> +	int x1 = x * 640/80;
> +	int c = buf[fmt.fmt.pix.width*3*y1 + 3*x1] +
> +	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 1] +
> +	  buf[fmt.fmt.pix.width*3*y1 + 3*x1 + 2];
> +
> +	if (c < 30) c = ' ';
> +	else if (c < 60) c = '.';
> +	else if (c < 120) c = '_';
> +	else if (c < 180) c = 'o';
> +	else if (c < 300) c = 'x';
> +	else if (c < 400) c = 'X';
> +	else c = '#';
> +	  
> +	printf("%c", c);
> +      }
> +      printf("\n");
> +    }

IMHO, it would be better to use aalib. Btw, xawtv3 has a code example
using it, under:
	console/ttv.c

As it already uses libv4l, prhaps you could use it, instead of adding
a new ascii app.

> +#endif    
> +  }
> +}
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..af740a7 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */
>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -70,6 +71,14 @@
>  	} while (0)
>  
>  #define MIN(a, b) (((a) < (b)) ? (a) : (b))
> +#define V4L2_MAX_SUBDEVS 16
> +
> +#define V4L2_MAX_FOCUS 10
> +struct v4l2_focus_step {
> +	int num;
> +	int sharpness[V4L2_MAX_FOCUS];
> +	int position[V4L2_MAX_FOCUS];
> +};
>  
>  struct v4l2_dev_info {
>  	int fd;
> @@ -104,6 +113,14 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
> +  	int exposure; 
> +  	int frame;
> +  	int focus;
> +
> +	/* Autofocus parameters */
> +	int focus_frame, focus_exposure, focus_step, sh_prev;
> +	struct v4l2_focus_step focus_step_params;
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..3b84437 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */

Don't add editor-specific macros inside the code. Not everybody uses emacs.

>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -100,6 +101,13 @@ static struct v4l2_dev_info devices[V4L2_MAX_DEVICES] = {
>  };
>  static int devices_used;
>  
> +#include "sdl.c"
> +
> +static struct sdl sdl;
> +
> +int v4l2_get_index(int fd);
> +void my_main(void);
> +

The above looks really odd. Why do you want to make libv4l2 dependent
on sdl?

>  static int v4l2_ensure_convert_mmap_buf(int index)
>  {
>  	if (devices[index].convert_mmap_buf != MAP_FAILED) {
> @@ -311,7 +319,409 @@ static int v4l2_queue_read_buffer(int index, int buffer_index)
>  	return 0;
>  }
>  
> -static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
> +static void v4l2_paint(char *buf, int x1, int y1, const struct v4l2_format *fmt)
> +{
> +  int y, x;
> +  int width = fmt->fmt.pix.width;
> +  printf("width = %d\n", width);
> +    
> +    for (y = 0; y < 25; y++) {
> +      for (x = 0; x < 80; x++) {
> +	int c = buf[width*4*y1 + 4*x1] +
> +	  buf[width*4*y1 + 4*x1 + 1] +
> +	  buf[width*4*y1 + 4*x1 + 2];
> +
> +	if (c < 30) c = ' ';
> +	else if (c < 60) c = '.';
> +	else if (c < 120) c = '_';
> +	else if (c < 180) c = 'o';
> +	else if (c < 300) c = 'x';
> +	else if (c < 400) c = 'X';
> +	else c = '#';
> +	  
> +	printf("%c", c);
> +      }
> +      printf("\n");
> +    }
> +}

Please don't add anything like that inside the library.

> +
> +#define SX 80
> +#define SY 25
> +#define BUCKETS 20
> +
> +static void v4l2_histogram(unsigned char *buf, int cdf[], struct v4l2_format *fmt)
> +{
> +    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
> +      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
> +	pixel p = buf_pixel(fmt, buf, x, y);
> +	
> +	int b;
> +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> +	b = p.r + 2*p.g + p.b;
> +	b = (b * BUCKETS)/(256);
> +	cdf[b]++;
> +      }
> +}
> +
> +static long v4l2_sharpness(unsigned char *buf, struct v4l2_format *fmt)
> +{
> +  int h = fmt->fmt.pix.height;
> +  int w = fmt->fmt.pix.width;
> +  long r = 0;
> +
> +    for (int y = h/3; y < h-h/3; y+=h/9)
> +      for (int x = w/3; x < w-w/3; x++) {
> +	pixel p1 = buf_pixel(fmt, buf, x, y);
> +	pixel p2 = buf_pixel(fmt, buf, x+2, y);
> +	
> +	int b1, b2;
> +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> +	b1 = p1.r + 2*p1.g + p1.b;
> +	b2 = p2.r + 2*p2.g + p2.b;
> +
> +	int v;
> +	v = (b1-b2)*(b1-b2);
> +	if (v > 36)
> +		r+=v;
> +      }
> +
> +    return r;
> +}

IMO, the above belongs to a separate processing module under
	lib/libv4lconvert/processing/

> +
> +int v4l2_set_exposure(int fd, int exposure)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +
> +	struct v4l2_control ctrl;
> +	ctrl.id = V4L2_CID_EXPOSURE;
> +	ctrl.value = exposure;
> +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> +	  printf("Could not set exposure\n");
> +	}
> +	return 0;
> +}

Shouldn't it be together with lib/libv4lconvert/processing/autogain.c,
perhaps as an alternative implementation, if what's there is not enough?

> +
> +int v4l2_set_gain(int fd, int gain)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +	
> +	struct v4l2_control ctrl;
> +	ctrl.id = 0x00980913;

Don't hardcode control numbers here.

> +	ctrl.value = gain;
> +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> +	  printf("Could not set exposure\n");
> +	}
> +	return 0;
> +}
> +
> +int v4l2_set_focus(int fd, int diopt)
> +{
> +	int index = v4l2_get_index(fd);
> +
> +	if (index == -1 || devices[index].convert == NULL) {
> +		V4L2_LOG_ERR("v4l2_set_focus called with invalid fd: %d\n", fd);
> +		errno = EBADF;
> +		return -1;
> +	}
> +
> +	struct v4l2_control ctrl;
> +	ctrl.id = V4L2_CID_FOCUS_ABSOLUTE;
> +	ctrl.value = diopt;
> +	if (ioctl(devices[index].subdev_fds[1], VIDIOC_S_CTRL, &ctrl) < 0) {
> +		printf("Could not set focus\n");
> +	}
> +	return 0;
> +}
> +
> +#define LESS 1
> +#define MID 0
> +#define MORE 2
> +
> +static void v4l2_start_focus_step(struct v4l2_focus_step *fs, int focus, int step)
> +{
> +	int i;
> +
> +	fs->num = 3;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->sharpness[i] = -1;
> +		fs->position[i] = -1;
> +	}
> +
> +	fs->position[LESS] = focus - step;
> +	fs->position[MID] = focus;
> +	fs->position[MORE] = focus + step;
> +}
> +
> +static int v4l2_focus_get_best(struct v4l2_focus_step *fs)
> +{
> +	int i, max = -1, maxi = -1;
> +	
> +	for (i=0; i<fs->num; i++)
> +		if (max < fs->sharpness[i]) {
> +			max = fs->sharpness[i];
> +			maxi = i;
> +		}
> +
> +	return maxi;
> +}
> +
> +static void v4l2_auto_focus_continuous(struct v4l2_dev_info *m, int sh)
> +{
> +	int f = m->frame - m->focus_frame;
> +	int step;
> +	const int max_step = 300, min_step = 20;
> +	struct v4l2_focus_step *fs = &m->focus_step_params;
> +
> +	if (m->focus_step == 0 || m->focus_step > max_step) {
> +		printf("step reset -- max\n");
> +		m->focus_step = max_step;
> +	}
> +	if (m->focus_step < min_step) {
> +		printf("step reset -- 10 (%d)\n", m->focus_step);
> +		m->focus_step = min_step;
> +		/* It takes cca 5.7 seconds to achieve the focus:
> +		   0.76user 0.30system 5.66 (0m5.661s) elapsed 18.72%CPU
> +		 */

The above note seems hardware dependent. 

> +		printf("Focused at %d\n", m->focus);
> +		exit(0);
> +	}
> +
> +	step = m->focus_step;
> +
> +	if (m->exposure != m->focus_exposure) {
> +		m->focus_frame = m->frame;
> +		m->focus_exposure = m->exposure;
> +		v4l2_start_focus_step(fs, m->focus, m->focus_step);
> +		return;
> +	}
> +	if (m->focus < step) {
> +		m->focus = step;
> +	}
> +
> +	const int every = 3;
> +	if (f%every)
> +		return;
> +
> +	{
> +		int i = f/every;
> +
> +		if (i == 0) {
> +			printf("Can not happen?\n");
> +			return;
> +		}
> +		i--;
> +		if (i < fs->num)
> +			v4l2_set_focus(m->fd, fs->position[i]);
> +		if (i > 0)
> +			fs->sharpness[i-1] = sh;
> +		if (i < fs->num)
> +			return;
> +	}
> +	int i;
> +	for (i=0; i<fs->num; i++) {
> +		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
> +	}

You should probably print something only if log is enabled. Take a look
at lib/libv4l2/log.c. Same applies to similar printf() calls.

> +	int best = v4l2_focus_get_best(fs);
> +	if ((fs->sharpness[best] < m->sh_prev) && (m->focus_step < max_step)) {
> +		m->focus_step *=2;
> +		m->sh_prev = m->sh_prev * 0.9;
> +		printf("step up %d\n", m->focus_step);
> +	} else if (best == LESS) {
> +		printf("less ");
> +		m->focus -= step;
> +	} else if (best == MORE) {
> +		printf("more ");
> +		m->focus += step;
> +	} else {
> +		m->sh_prev = fs->sharpness[MID];
> +		m->focus_step = m->focus_step / 2;
> +		printf("step %d ", m->focus_step);
> +	}
> +	m->focus_frame = m->frame;
> +	v4l2_start_focus_step(fs, m->focus, m->focus_step);
> +	printf("Focus now %d\n", m->focus);
> +}
> +
> +static void v4l2_start_focus_sweep(struct v4l2_focus_step *fs, int focus, int step)
> +{
> +	int i;
> +
> +	fs->num = V4L2_MAX_FOCUS;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->sharpness[i] = -1;
> +		fs->position[i] = -1;
> +	}
> +
> +	int f = focus;
> +	for (i=0; i<V4L2_MAX_FOCUS; i++) {
> +		fs->position[i] = f;
> +		f += step;
> +	}
> +}
> +
> +static void v4l2_auto_focus_single(struct v4l2_dev_info *m, int sh)
> +{
> +	int f = m->frame - m->focus_frame;
> +	int step;
> +	struct v4l2_focus_step *fs = &m->focus_step_params;
> +
> +	if (m->focus_step == 0) {
> +		printf("step reset -- max\n");
> +		m->focus_step = 1;
> +	}
> +
> +	if (m->exposure != m->focus_exposure) {
> +		m->focus_frame = m->frame;
> +		m->focus_exposure = m->exposure;
> +		v4l2_start_focus_sweep(fs, 0, 100);
> +		return;
> +	}
> +
> +	const int every = 3;
> +	if (f%every)
> +		return;
> +
> +	{
> +		int i = f/every;
> +
> +		if (i == 0) {
> +			printf("Can not happen?\n");
> +			return;
> +		}
> +		i--;
> +		if (i < fs->num)
> +			v4l2_set_focus(m->fd, fs->position[i]);
> +		if (i > 0)
> +			fs->sharpness[i-1] = sh;
> +		if (i < fs->num)
> +			return;
> +	}
> +#if 0
> +	int i;
> +	for (i=0; i<fs->num; i++) {
> +		printf("%d: %d | ", fs->position[i], fs->sharpness[i]);
> +	}
> +#endif
> +	int best = v4l2_focus_get_best(fs);
> +	m->focus_frame = m->frame;
> +	switch (m->focus_step) {
> +	case 1:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		v4l2_start_focus_sweep(fs, fs->position[best] - 50, 10);
> +		m->focus_step = 2;
> +		break;
> +	case 2:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		v4l2_start_focus_sweep(fs, fs->position[best] - 10, 2);
> +		m->focus_step = 3;
> +		break;
> +	case 3:
> +		printf("Best now %d / %d\n", fs->position[best], fs->sharpness[best]);
> +		printf("done.\n");
> +		exit(0);
> +	}
> +}
> +
> +
> +static void v4l2_auto_exposure(int index, struct v4l2_buffer *buf)
> +{
> +	struct v4l2_format *fmt;
> +	int cdf[BUCKETS] = { 0, };
> +	int i;
> +
> +	fmt = &devices[index].src_fmt;
> +
> +	v4l2_histogram(devices[index].frame_pointers[buf->index], cdf, fmt);
> +
> +#if 0
> +	printf("hist: ");
> +	for (i = 0; i<BUCKETS; i++)
> +		printf("%d ", cdf[i]);
> +	printf("\n");
> +#endif
> +	for (i=1; i<BUCKETS; i++)
> +		cdf[i] += cdf[i-1];
> +
> +	int b = BUCKETS;
> +	int brightPixels = cdf[b-1] - cdf[b-8];
> +	int targetBrightPixels = cdf[b-1]/50;
> +	int maxSaturatedPixels = cdf[b-1]/200;
> +	int saturatedPixels = cdf[b-1] - cdf[b-2];
> +	// how much should I change brightness by
> +	float adjustment = 1.0f;
> +#if 0
> +	printf( "AutoExposure: totalPixels: %d,"
> +		"brightPixels: %d, targetBrightPixels: %d,"
> +		"saturatedPixels: %d, maxSaturatedPixels: %d\n",
> +		cdf[b-1], brightPixels, targetBrightPixels,
> +		saturatedPixels, maxSaturatedPixels);
> +#endif
> +	  
> +	if (saturatedPixels > maxSaturatedPixels) {
> +		// first don't let things saturate too much
> +		adjustment = 1.0f - ((float)(saturatedPixels - maxSaturatedPixels))/cdf[b-1];
> +	} else if (brightPixels < (targetBrightPixels - (saturatedPixels * 4))) {
> +		// increase brightness to try and hit the desired number of well exposed pixels
> +		int l = b-6;
> +		while (brightPixels < targetBrightPixels && l > 0) {
> +			brightPixels += cdf[l];
> +			brightPixels -= cdf[l-1];
> +			l--;
> +		}
> +
> +		// that level is supposed to be at b-11;
> +		adjustment = ((float) (b-6+1))/(l+1);
> +	} else {
> +		// we're not oversaturated, and we have enough bright pixels. Do nothing.
> +	}
> +
> +	{
> +		float limit = 4;
> +		if (adjustment > limit) { adjustment = limit; }
> +		if (adjustment < 1/limit) { adjustment = 1/limit; }
> +	}
> +	  
> +	if (!devices[index].exposure)
> +		devices[index].exposure = 1;
> +	devices[index].exposure *= adjustment;
> +	if (adjustment != 1.)
> +		printf( "AutoExposure: adjustment: %f exposure %d\n", adjustment, devices[index].exposure);
> +
> +	v4l2_set_exposure(devices[index].fd, devices[index].exposure);
> +}
> +
> +static void v4l2_statistics(int index, struct v4l2_buffer *buf)
> +{
> +	unsigned char *b;
> +	struct v4l2_format *fmt;
> +
> +	fmt = &devices[index].src_fmt;
> +	b = devices[index].frame_pointers[buf->index];
> +
> +	if (!(devices[index].frame%3))
> +		v4l2_auto_exposure(index, buf);
> +	
> +	int sh = v4l2_sharpness(b, fmt);
> +	v4l2_auto_focus_single(&devices[index], sh);
> +	
> +	devices[index].frame++;
> +	if (!(devices[index].frame%4))
> +		sdl_render(&sdl, b, fmt);
> +}
> +
> +int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  		unsigned char *dest, int dest_size)
>  {
>  	const int max_tries = V4L2_IGNORE_FIRST_FRAME_ERRORS + 1;
> @@ -345,6 +755,13 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  			errno = -EINVAL;
>  			return -1;
>  		}
> +		
> +#if 1
> +		v4l2_statistics(index, buf);
> +#endif
> +#if 0
> +		/* This is rather major eater of CPU time. CPU time goes from 80% to 4% 
> +		   when conversion is disabled. */
>  
>  		result = v4lconvert_convert(devices[index].convert,
>  				&devices[index].src_fmt, &devices[index].dest_fmt,
> @@ -352,7 +769,7 @@ static int v4l2_dequeue_and_convert(int index, struct v4l2_buffer *buf,
>  				buf->bytesused, dest ? dest : (devices[index].convert_mmap_buf +
>  					buf->index * devices[index].convert_mmap_frame_size),
>  				dest_size);
> -
> +#endif
>  		if (devices[index].first_frame) {
>  			/* Always treat convert errors as EAGAIN during the first few frames, as
>  			   some cams produce bad frames at the start of the stream
> @@ -789,18 +1206,24 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +
> +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> +	       devices[index].subdev_fds[1]);
> +
>  	V4L2_LOG("open: %d\n", fd);
>  
>  	return fd;
>  }
>  
>  /* Is this an fd for which we are emulating v4l1 ? */
> -static int v4l2_get_index(int fd)
> +int v4l2_get_index(int fd)
>  {
>  	int index;
>  
> @@ -823,6 +1246,10 @@ int v4l2_close(int fd)
>  {
>  	int index, result;
>  
> +	if (fd == -2) {
> +	  my_main();
> +	}
> +

That looks a hack!

>  	index = v4l2_get_index(fd);
>  	if (index == -1)
>  		return SYS_CLOSE(fd);
> @@ -1782,3 +2209,65 @@ int v4l2_get_control(int fd, int cid)
>  			(qctrl.maximum - qctrl.minimum) / 2) /
>  		(qctrl.maximum - qctrl.minimum);
>  }
> +
> +void v4l2_debug(void)
> +{
> +	printf("debug\n");
> +}
> +
> +/* ------------------------------------------------------------------ */
> +
> +#define SIZE 10*1024*1024
> +
> +char buf[SIZE];
> +
> +void my_main(void)
> +{
> +	int fd = v4l2_open("/dev/video2", O_RDWR);
> +	int i;
> +	static struct v4l2_format fmt;
> +
> +	fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
> +	fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
> +	fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
> +	fmt.fmt.pix.width = 640;
> +	fmt.fmt.pix.height = 480;
> +
> +	v4l2_set_gain(fd, 300);
> +	v4l2_set_exposure(fd, 10000);
> +	v4l2_set_focus(fd, 0);
> +
> +
> +	printf("ioctl = %d\n", v4l2_ioctl(fd, VIDIOC_S_FMT, &fmt));
> +
> +	printf("capture is %d, %d\n", fmt.fmt.pix.width, fmt.fmt.pix.height);
> +	/* factor == 2 still fits window, but very slow. factor == 6 .. */
> +	/* factor needs to be odd, otherwise ... fun with BA10 format. */
> +	sdl_init(&sdl, fmt.fmt.pix.width, fmt.fmt.pix.height, 5);
> +  
> +	v4l2_debug();
> +
> +	/* In 800x600 "raw" mode, this should take cca 17.8 seconds (without
> +	   sdl output. CPU usage should be cca 5% without conversion). That's 28 fps.
> +	   (benchmark with i<500)
> +	*/
> +	for (i=0; i<50000; i++) {
> +		int num = v4l2_read(fd, buf, SIZE);
> +
> +		if (i==490) {
> +			printf("Focus to closest.... -------------------\n");
> +			v4l2_set_focus(fd, 99999);
> +		}
> +    
> +#if 0
> +		v4l2_paint(buf, 640/80, 480/25, &fmt);
> +#endif
> +		/* Over USB connection, rendering every single frame slows
> +		   execution down from 23 seconds to 36 seconds. */
> +#if 0
> +		if (!(i%4))
> +			sdl_render(&sdl, buf, &fmt);
> +#endif
> +	}
> +  
> +}
> diff --git a/lib/libv4l2/sdl.c b/lib/libv4l2/sdl.c
> new file mode 100644
> index 0000000..17a8d24
> --- /dev/null
> +++ b/lib/libv4l2/sdl.c
> @@ -0,0 +1,530 @@
> +/* -*- c-file-style: "linux" -*- */
> +/* SDL support.
> +
> +   Copyright 2017 Pavel Machek, LGPL
> +*/
> +
> +#include <SDL2/SDL.h>
> +#include <SDL2/SDL_image.h>

If you're adding a SDL-specific application, you'll need to add the 
needed autoconf bits to detect if SDL devel package is installed,
auto-disabling it if not.

Yet, I don't think that SDL should be part of the library, but,
instead, part of some application.

> +
> +struct sdl {
> +	SDL_Window *window;
> +	SDL_Surface *liveview, *screen;
> +
> +	int wx, wy;
> +	int sx, sy;
> +	int bx, by;
> +	int factor;
> +	float focus, gain, exposure, do_focus, do_exposure; 
> +};
> +
> +#if 0
> +void loop(void) {
> +	int done;
> +	SDL_Event event;
> +
> +	while(!done){ //While program isn't done                                   
> +		while(SDL_PollEvent(&event)){ //Poll events                        
> +			switch(event.type){ //Check event type                     
> +			case SDL_QUIT: //User hit the X (or equivelent)            
> +				done = true; //Make the loop end                   
> +				break; //We handled the event                      
> +			} //Finished with current event                            
> +		} //Done with all events for now                                   
> +	} //Program done, exited                                                   
> +}
> +#endif
> +
> +typedef struct {
> +	Uint8 r;
> +	Uint8 g;
> +	Uint8 b;
> +	Uint8 alpha;
> +} pixel;
> +
> +#define d_raw 1
> +
> +void sfc_put_pixel(SDL_Surface* liveview, int x, int y, pixel *p)
> +{
> +	Uint32* p_liveview = (Uint32*)liveview->pixels;
> +	p_liveview += y*liveview->w+x;
> +	*p_liveview = SDL_MapRGBA(liveview->format,p->r,p->g,p->b,p->alpha);
> +}
> +
> +#if 0
> +int pix_exposure(float x)
> +{
> +	return sy*x / 199410.0;
> +}
> +
> +int pix_gain(float x)
> +{
> +	return sy*x / 16.0;
> +}
> +
> +int render_statistics(SDL_Surface* liveview)
> +{
> +	pixel white;
> +	white.r = (Uint8)0xff;
> +	white.g = (Uint8)0xff;
> +	white.b = (Uint8)0xff;
> +	white.alpha = (Uint8)128;
> +
> +	//printf("Stats: focus %d, gain %d, exposure %d\n", focus, gain, exposure);
> +
> +	for (int x=0; x<sx && x<sx*focus; x++)
> +		put_pixel(liveview, x, 0, &white);
> +
> +	for (int y=0; y<sy && y<pix_gain(gain); y++)
> +		put_pixel(liveview, 0, y, &white);
> +
> +	for (int y=0; y<sy && y<pix_exposure(exposure); y++)
> +		put_pixel(liveview, sx-1, y, &white);
> +
> +	for (int x=0; x<sx; x++)
> +		put_pixel(liveview, x, sy-1, &white);
> +
> +	return 0;
> +}
> +#endif
> +
> +void sdl_begin_paint(struct sdl *m) {
> +	//Fill the surface white                                                   
> +	SDL_FillRect(m->liveview, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
> +
> +	SDL_LockSurface(m->liveview);
> +}
> +
> +void sdl_finish_paint(struct sdl *m) {
> +	SDL_UnlockSurface(m->liveview);
> +	SDL_Rect rcDest = { m->bx, m->by, m->sx, m->sy };
> +
> +	SDL_BlitSurface(m->liveview, NULL, m->screen, &rcDest);
> +	//Update the surface                                                       
> +	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
> +}
> +  
> +void sdl_paint_image(struct sdl *m, char **xpm, int x, int y) {
> +	SDL_Surface *image = IMG_ReadXPMFromArray(xpm);
> +	if (!image) {
> +		printf("IMG_Load: %s\n", IMG_GetError());
> +		exit(1);
> +	}
> +
> +	int x_pos = x - image->w/2, y_pos = y - image->h/2;
> +
> +	SDL_Rect rcDest = { x_pos, y_pos, image->w, image->h };
> +	int r = SDL_BlitSurface ( image, NULL, m->screen, &rcDest );
> +
> +	if (r) {
> +		printf("Error blitting: %s\n", SDL_GetError());
> +		exit(1);
> +	}
> +	SDL_FreeSurface ( image );
> +}
> +
> +void sdl_paint_ui(struct sdl *m) {
> +	static char *wait_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"....########....",
> +		".....#....#.....",
> +		".....#....#.....",
> +		"......#..#......",
> +		".......##.......",
> +		"......#..#......",
> +		".....#....#.....",
> +		".....#....#.....",
> +		"....########....",
> +	};
> +
> +	static char *ok_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"...............#",
> +		"............###.",
> +		"..........##....",
> +		"#.......##......",
> +		".#.....#........",
> +		"..#...#.........",
> +		"..#..#..........",
> +		"...##...........",
> +		"...#............",
> +	};
> +
> +	static char *exit_xpm[] = {
> +		"16 9 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"....x......x....",
> +		".....x....x.....",
> +		"......x..x......",
> +		".......xx.......",
> +		".......xx.......",
> +		"......x..x......",
> +		".....x....x.....",
> +		"....x......x....",
> +		"................",
> +	};
> +
> +	static char *f1m_xpm[] = {
> +		"16 9 2 1",
> +		"# c #ffffff",
> +		". c #000000",
> +		"....##..........",
> +		"...#.#..........",
> +		"..#..#..........",
> +		".....#...#.#.##.",
> +		".....#...##.#..#",
> +		".....#...#..#..#",
> +		".....#...#..#..#",
> +		".....#...#..#..#",
> +		"................",
> +	};
> +
> +	static char *f25cm_xpm[] = {
> +		"16 9 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".xxx..xxxx......",
> +		"x...x.x.........",
> +		"...x..xxx.......",
> +		"..x......x..xx.x",
> +		".x.......x.x.xxx",
> +		"xxxxx.xxx...xxxx",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	static char *iso400_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"x..x.xxxx.xxxx..",
> +		"x..x.x..x.x..x..",
> +		"xxxx.x..x.x..x..",
> +		"...x.x..x.x..x..",
> +		"...x.xxxx.xxxx..",
> +		"................",
> +		".x..xx..x.......",
> +		".x.x...x.x......",
> +		".x..x..x.x......",
> +		".x...x.x.x......",
> +		".x.xx...x.......",
> +		"................",
> +	};
> +
> +	static char *time_1_100_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".x....x.........",
> +		".x...x..........",
> +		".x..x...........",
> +		".x.x............",
> +		"................",
> +		"..x.xxxx.xxxx...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.x..x.x..x...",
> +		"..x.xxxx.xxxx...",
> +		"................",
> +	};
> +
> +	static char *time_1_10_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".x....x.........",
> +		".x...x..........",
> +		".x..x...........",
> +		".x.x............",
> +		"................",
> +		"..x.xxxx........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.x..x........",
> +		"..x.xxxx........",
> +		"................",
> +	};
> +
> +	static char *af_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".....xxxxxxx....",
> +		".....x..........",
> +		".....x..........",
> +		".x...xxxxx......",
> +		"x.x..x..........",
> +		"xxx..x..........",
> +		"x.x..x..........",
> +		"x.x..x..........",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	static char *ae_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		".....xxxxxxx....",
> +		".....x..........",
> +		".....x..........",
> +		".x...xxxxx......",
> +		"x.x..x..........",
> +		"xxx..x..........",
> +		"x.x..x..........",
> +		"x.x..xxxxxxx....",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +    
> +	static char *not_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"......xxxxx.....",
> +		"....xx.....xx...",
> +		"...x.........x..",
> +		"..x........xx.x.",
> +		"..x......xx...x.",
> +		".x.....xx......x",
> +		".x...xx........x",
> +		"..xxx.........x.",
> +		"..x...........x.",
> +		"...x.........x..",
> +		"....xx.....xx...",
> +		"......xxxxx.....",
> +	};
> +
> +	static char *empty_xpm[] = {
> +		"16 12 2 1",
> +		"x c #ffffff",
> +		". c #000000",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +		"................",
> +	};
> +
> +	SDL_FillRect(m->screen, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
> +	sdl_paint_image(m, wait_xpm,  m->wx/2,     m->wy/2);
> +	sdl_paint_image(m, ok_xpm,    m->wx-m->bx/2,  m->wy-m->by/2);
> +	sdl_paint_image(m, exit_xpm,  m->bx/2,     m->wy-m->by/2);
> +	sdl_paint_image(m, f1m_xpm,   m->bx+m->sx/20, m->by/2);
> +	sdl_paint_image(m, f25cm_xpm, m->bx+m->sx/5,  m->by/2);
> +
> +	sdl_paint_image(m, af_xpm,    m->bx/2,     m->by/2);
> +	if (!m->do_focus) {
> +		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
> +	}
> +	sdl_paint_image(m, ae_xpm,    m->wx-m->bx/2,  m->by/2);
> +	if (!m->do_exposure) {
> +		sdl_paint_image(m, not_xpm,  16+m->bx/2,     m->by/2);      
> +	}
> +
> +#if 0
> +	sdl_paint_image(m, time_1_100_xpm, m->wx-m->bx/2, m->by+pix_exposure(10000));
> +	sdl_paint_image(m, time_1_10_xpm,  m->wx-m->bx/2, m->by+pix_exposure(100000));
> +	sdl_paint_image(m, iso400_xpm,     m->bx/2,    m->by+pix_gain(4.0));
> +#endif
> +		
> +	SDL_UpdateWindowSurface(m->window);
> +}
> +
> +void fmt_print(struct v4l2_format *fmt)
> +{
> +	int f;
> +	printf("Format: %dx%d. ", fmt->fmt.pix.width, fmt->fmt.pix.height);
> +	printf("%x ", fmt->fmt.pix.pixelformat);
> +	f = fmt->fmt.pix.pixelformat;
> +	for (int i=0; i<4; i++) {
> +		printf("%c", f & 0xff);
> +		f >>= 8;
> +	}
> +	printf("\n");
> +}
> +
> +pixel buf_pixel(struct v4l2_format *fmt, unsigned char *buf, int x, int y)
> +{
> +	pixel p = { 0, 0, 0, 0 };
> +	int pos = x + y*fmt->fmt.pix.width;
> +	int b;
> +
> +	p.alpha = 128;
> +
> +	switch (fmt->fmt.pix.pixelformat) {
> +	case '01AB': /* BA10, aka GRBG10, 
> +			https://www.linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-srggb10.html?highlight=ba10
> +		     */
> +		b = buf[pos*2];
> +		b += buf[pos*2+1] << 8;
> +		b /= 4;
> +		if ((y % 2) == 0) {
> +			switch (x % 2) {
> +			case 0: p.g = b/2; break;
> +			case 1: p.r = b; break;
> +			}
> +		} else {
> +			switch (x % 2) {
> +			case 0: p.b = b; break;
> +			case 1: p.g = b/2; break;
> +			}
> +		}
> +		break;
> +
> +	case V4L2_PIX_FMT_RGB24:
> +		pos *= 4;
> +		p.r = buf[pos];
> +		p.g = buf[pos+1];
> +		p.b = buf[pos+2];
> +		break;
> +
> +	default:
> +		printf("Wrong pixel format!\n");
> +		fmt_print(fmt);
> +		exit(1);
> +	}
> +
> +	return p;
> +}
> +
> +void sdl_render(struct sdl *m, unsigned char *buf, struct v4l2_format *fmt)
> +{
> +	if (!m->window) 
> +		return;
> +	sdl_begin_paint(m);    
> +	/* do your rendering here */
> +
> +	for (int y = 0; y < m->sy; y++)
> +		for (int x = 0; x < m->sx; x++) {
> +			pixel p = buf_pixel(fmt, buf, x*m->factor, y*m->factor);
> +			p.alpha = 128;
> +			sfc_put_pixel(m->liveview, x, y, &p);
> +		}
> +
> +	//render_statistics(liveview);
> +
> +	sdl_finish_paint(m);
> +}
> +
> +#if 0
> +void imageStatistics(FCam::Image img)
> +{
> +	int sx, sy;
> +
> +	sx = img.size().width;                                                          
> +	sy = img.size().height;                                                         
> +	printf("Image is %d x %d, ", sx, sy);                                           
> +
> +	printf("(%d) ", img.brightness(sx/2, sy/2));
> +
> +	int dark = 0;                                                                   
> +	int bright = 0;                                                                 
> +	int total = 0;                                                                  
> +#define RATIO 10
> +	for (int y = sy/(2*RATIO); y < sy; y += sy/RATIO)                               
> +		for (int x = sx/(2*RATIO); x < sx; x += sx/RATIO) {                    
> +			int br = img.brightness(x, y);                               
> +
> +			if (!d_raw) {
> +				/* It seems real range is 60 to 218 */
> +				if (br > 200)
> +					bright++;                                               
> +				if (br < 80)
> +					dark++;
> +			} else {
> +				/* there's a lot of noise, it seems black is commonly 65..71,
> +				   bright is cca 1023 */
> +				if (br > 1000)
> +					bright++;
> +				if (br < 75)
> +					dark++;
> +			}
> +			total++;                                                      
> +		}
> +
> +	printf("%d dark %d bri,", dark, bright);                     
> +
> +	long sharp = 0;
> +	for (int y = sy/3; y < 2*(sy/3); y+=sy/12) {
> +		int b = -1;
> +		for (int x = sx/3; x < 2*(sx/3); x++) {
> +			int b2;                                                                
> +			b2 = img.brightness(x, y/2);                                          
> +			if (b != -1)
> +				sharp += (b-b2) * (b-b2);                                              
> +			b = b2;                                                                
> +		}
> +	}
> +	printf("sh %d\n", sharp);
> +}
> +#endif
> +
> +
> +void sdl_init(struct sdl *m, int _sx, int _sy, int _factor)
> +{
> +	printf("Initing SDL\n");
> +
> +	if (SDL_Init(SDL_INIT_VIDEO) < 0) {
> +		printf("Could not init SDL\n");
> +		exit(1);
> +	}
> +
> +	atexit(SDL_Quit);
> +
> +	m->wx = 800;
> +	m->wy = 400;
> +
> +	m->window = SDL_CreateWindow( "Camera", SDL_WINDOWPOS_UNDEFINED,
> +				      SDL_WINDOWPOS_UNDEFINED, m->wx, m->wy,
> +				      SDL_WINDOW_SHOWN );
> +	if (m->window == NULL) {
> +		printf( "Window could not be created! SDL_Error: %s\n", SDL_GetError() );
> +	}
> +
> +	m->screen = SDL_GetWindowSurface(m->window);
> +	if (!m->screen) {
> +		printf("Couldn't create liveview\n");
> +		exit(1);
> +	}
> +
> +	m->sx = _sx;
> +	m->sy = _sy;
> +	m->factor = _factor;
> +
> +	m->sx /= m->factor;
> +	m->sy /= m->factor;
> +    
> +	m->focus = 0;
> +	m->gain = 0;
> +	m->exposure = 0;
> +
> +	m->bx = (m->wx-m->sx)/2;
> +	m->by = (m->wy-m->sy)/2;
> +
> +	m->liveview = SDL_CreateRGBSurface(0,m->sx,m->sy,32,0,0,0,0);
> +	if (!m->liveview) {
> +		printf("Couldn't create liveview\n");
> +		exit(1);
> +	}
> +	sdl_paint_ui(m);
> +}
> diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
> index d3d8936..7521ec8 100644
> --- a/lib/libv4lconvert/libv4lconvert.c
> +++ b/lib/libv4lconvert/libv4lconvert.c
> @@ -1205,6 +1205,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
>  			v4lconvert_swap_uv(src, dest, fmt);
>  			break;
>  		}
> +		default:
> +		  /* This is bad, fixme? */
>  		break;
>  
>  	case V4L2_PIX_FMT_YVU420:
> @@ -1228,6 +1230,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
>  		case V4L2_PIX_FMT_YVU420:
>  			memcpy(dest, src, width * height * 3 / 2);
>  			break;
> +		default:
> +		  /* This is bad, fixme? */
>  		}
>  		break;
>  
> diff --git a/utils/decode_tm6000/Makefile.am b/utils/decode_tm6000/Makefile.am
> index ac4e85e..f059e3c 100644
> --- a/utils/decode_tm6000/Makefile.am
> +++ b/utils/decode_tm6000/Makefile.am
> @@ -1,4 +1,4 @@
>  bin_PROGRAMS = decode_tm6000
>  decode_tm6000_SOURCES = decode_tm6000.c
> -decode_tm6000_LDADD = ../libv4l2util/libv4l2util.la
> +decode_tm6000_LDADD = ../libv4l2/libv4l2.la
>  decode_tm6000_LDFLAGS = $(ARGP_LIBS)
> diff --git a/utils/decode_tm6000/decode_tm6000.c b/utils/decode_tm6000/decode_tm6000.c
> index 4bffbdd..fda7e3b 100644
> --- a/utils/decode_tm6000/decode_tm6000.c
> +++ b/utils/decode_tm6000/decode_tm6000.c

Everything below it is completely wrong!

> @@ -1,365 +1,16 @@
> -/*
> -   decode_tm6000.c - decode multiplexed format from TM5600/TM6000 USB
> +/* gcc asciicam.c /usr/lib/i386-linux-gnu/libv4l2.so.0.0.0 -o asciicam
> +   gcc asciicam.c /usr/lib/arm-linux-gnueabi/libv4l2.so.0 -o asciicam
>  
> -   Copyright (C) 2007 Mauro Carvalho Chehab <mchehab@infradead.org>
> -
> -   This program is free software; you can redistribute it and/or modify
> -   it under the terms of the GNU General Public License as published by
> -   the Free Software Foundation version 2.
> -
> -   This program is distributed in the hope that it will be useful,
> -   but WITHOUT ANY WARRANTY; without even the implied warranty of
> -   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> -   GNU General Public License for more details.
> -
> -   You should have received a copy of the GNU General Public License
> -   along with this program; if not, write to the Free Software
> -   Foundation, Inc., 51 Franklin Street, Suite 500, Boston, MA 02110-1335 USA.
> +gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../.. -fvisibility=hidden -I../../lib/include -Wall -Wpointer-arith -D_GNU_SOURCE -I../../include -g -O2 asciicam.c libv4l2.c /usr/lib/arm-linux-gnueabi/libv4lconvert.so.0 log.c v4l2convert.c v4l2-plugin.c -o asciicam
>   */
> -#include "../libv4l2util/v4l2_driver.h"
>  #include <stdio.h>
> -#include <string.h>
> -#include <argp.h>
> -#include <unistd.h>
>  #include <sys/types.h>
>  #include <sys/stat.h>
>  #include <fcntl.h>
> -#include <stdlib.h>
> -#include <errno.h>
> -
> -const char *argp_program_version="decode_tm6000 version 0.0.1";
> -const char *argp_program_bug_address="Mauro Carvalho Chehab <mchehab@infradead.org>";
> -const char doc[]="Decodes tm6000 proprietary format streams";
> -const struct argp_option options[] = {
> -	{"verbose",	'v',	0,	0,	"enables debug messages", 0},
> -	{"device",	'd',	"DEV",	0,	"uses device for reading", 0},
> -	{"output",	'o',	"FILE",	0,	"outputs raw stream to a file", 0},
> -	{"input",	'i',	"FILE",	0,	"parses a file, instead of a device", 0},
> -	{"freq",	'f',	"Freq",	0,	"station frequency, in MHz (default is 193.25)", 0},
> -	{"nbufs",	'n',	"quant",0,	"number of video buffers", 0},
> -	{"audio",	'a',	0,	0,	"outputs audio on stdout", 0},
> -	{"read",	'r',	0,	0,	"use read() instead of mmap method", 0},
> -	{ 0, 0, 0, 0, 0, 0 }
> -};
> -
> -static char outbuf[692224];
> -static int debug=0, audio=0, use_mmap=1, nbufs=4;
> -static float freq_mhz=193.25;
> -static char *devicename="/dev/video0";
> -static char *filename=NULL;
> -static enum {
> -	NORMAL,
> -	INPUT,
> -	OUTPUT
> -} mode = NORMAL;
> -
> -static FILE *fout;
> -
> -//const char args_doc[]="ARG1 ARG2";
> -
> -static error_t parse_opt (int key, char *arg, struct argp_state *state)
> -{
> -	switch (key) {
> -	case 'a':
> -		audio++;
> -		break;
> -	case 'r':
> -		use_mmap=0;
> -		break;
> -	case 'v':
> -		debug++;
> -		break;
> -	case 'd':
> -		devicename=arg;
> -		break;
> -	case 'i':
> -	case 'o':
> -		if (mode!=NORMAL) {
> -			argp_error(state,"You can't use input/output options simultaneously.\n");
> -			break;
> -		}
> -		if (key=='i')
> -			mode=INPUT;
> -		else
> -			mode=OUTPUT;
> -
> -		filename=arg;
> -		break;
> -	case 'f':
> -		freq_mhz=atof(arg);
> -		break;
> -	case 'n':
> -		nbufs=atoi(arg);
> -		if  (nbufs<2)
> -			nbufs=2;
> -		break;
> -	default:
> -		return ARGP_ERR_UNKNOWN;
> -	}
> -	return 0;
> -}
> -
> -static struct argp argp = {
> -	.options=options,
> -	.parser=parse_opt,
> -	.args_doc=NULL,
> -	.doc=doc,
> -};
>  
> -#define TM6000_URB_MSG_LEN 180
> -enum {
> -	TM6000_URB_MSG_VIDEO=1,
> -	TM6000_URB_MSG_AUDIO,
> -	TM6000_URB_MSG_VBI,
> -	TM6000_URB_MSG_PTS,
> -	TM6000_URB_MSG_ERR,
> -};
> -
> -const char *tm6000_msg_type[]= {
> -	"unknown(0)",	//0
> -	"video",	//1
> -	"audio",	//2
> -	"vbi",		//3
> -	"pts",		//4
> -	"err",		//5
> -	"unknown(6)",	//6
> -	"unknown(7)",	//7
> -};
> -
> -#define dprintf(fmt,arg...) \
> -	if (debug) fprintf(stderr, fmt, ##arg)
> -
> -static int recebe_buffer (struct v4l2_buffer *v4l2_buf, struct v4l2_t_buf *buf)
> -{
> -	dprintf("Received %zd bytes\n", buf->length);
> -fflush(stdout);
> -	memcpy (outbuf,buf->start,buf->length);
> -	return buf->length;
> -}
> -
> -
> -static int prepare_read (struct v4l2_driver *drv)
> -{
> -	struct v4l2_format fmt;
> -	double freq;
> -	int rc;
> -
> -	memset (drv,0,sizeof(*drv));
> -
> -	if (v4l2_open (devicename, 1,drv)<0) {
> -		perror ("Error opening dev");
> -		return -1;
> -	}
> -
> -	memset (&fmt,0,sizeof(fmt));
> -
> -	uint32_t pixelformat=V4L2_PIX_FMT_TM6000;
> -
> -	if (v4l2_gettryset_fmt_cap (drv,V4L2_SET,&fmt, 720, 480,
> -				    pixelformat,V4L2_FIELD_ANY)) {
> -		perror("set_input to tm6000 raw format");
> -		return -1;
> -	}
> -
> -	if (freq_mhz) {
> -		freq=freq_mhz * 1000 * 1000;
> -		rc=v4l2_getset_freq (drv,V4L2_SET, &freq);
> -		if (rc<0)
> -			printf ("Cannot set freq to %.3f MHz\n",freq_mhz);
> -	}
> -
> -	if (use_mmap) {
> -		printf("Preparing for receiving frames on %d buffers...\n",nbufs);
> -		fflush (stdout);
> -
> -		rc=v4l2_mmap_bufs(drv, nbufs);
> -		if (rc<0) {
> -			printf ("Cannot mmap %d buffers\n",nbufs);
> -			return -1;
> -		}
> -
> -//		v4l2_stop_streaming(&drv);
> -		rc=v4l2_start_streaming(drv);
> -		if (rc<0) {
> -			printf ("Cannot start streaming\n");
> -			return -1;
> -		}
> -	}
> -	printf("Waiting for frames...\n");
> -
> -	return 0;
> -}
> +#include <linux/videodev2.h>
>  
> -static int read_stream (struct v4l2_driver *drv, int fd)
> +void main(void)
>  {
> -	if (use_mmap) {
> -		fd_set fds;
> -		struct timeval tv;
> -		int r;
> -
> -		FD_ZERO (&fds);
> -		FD_SET (fd, &fds);
> -
> -		/* Timeout. */
> -		tv.tv_sec = 2;
> -		tv.tv_usec = 0;
> -
> -		r = select (fd + 1, &fds, NULL, NULL, &tv);
> -		if (-1 == r) {
> -			if (EINTR == errno) {
> -				perror ("select");
> -				return -errno;
> -			}
> -		}
> -
> -		if (0 == r) {
> -			fprintf (stderr, "select timeout\n");
> -			return -errno;
> -		}
> -
> -		return v4l2_rcvbuf(drv, recebe_buffer);
> -	} else {
> -		int size=read(fd, outbuf, sizeof(outbuf));
> -		return size;
> -	}
> -
> -	return 0;
> -}
> -
> -static int read_char (struct v4l2_driver *drv, int fd)
> -{
> -	static int sizebuf=0;
> -	static unsigned char *p=NULL;
> -	unsigned char c;
> -
> -	if (sizebuf<=0) {
> -		sizebuf=read_stream(drv,fd);
> -		if (sizebuf<=0)
> -			return -1;
> -		p=(unsigned char *)outbuf;
> -	}
> -	c=*p;
> -	p++;
> -	sizebuf--;
> -
> -	return c;
> -}
> -
> -
> -int main (int argc, char*argv[])
> -{
> -	int fd;
> -	unsigned int i;
> -	unsigned char buf[TM6000_URB_MSG_LEN], img[720*2*480];
> -	unsigned int  cmd, size, field, block, line, pos=0;
> -	unsigned long header=0;
> -	int           linesize=720*2,skip=0;
> -	struct v4l2_driver drv;
> -
> -	argp_parse (&argp, argc, argv, 0, 0, 0);
> -
> -	if (mode!=INPUT) {
> -		if (prepare_read (&drv)<0)
> -			return -1;
> -		fd=drv.fd;
> -	} else {
> -		/*mode == INPUT */
> -
> -		fd=open(filename,O_RDONLY);
> -		if (fd<0) {
> -			perror ("error opening a file for parsing");
> -			return -1;
> -		}
> -		dprintf("file %s opened for parsing\n",filename);
> -		use_mmap=0;
> -	}
> -
> -	if (mode==OUTPUT) {
> -		fout=fopen(filename,"w");
> -		if (!fout) {
> -			perror ("error opening a file to write");
> -			return -1;
> -		}
> -		dprintf("file %s opened for output\n",filename);
> -
> -		do {
> -			size=read_stream (&drv,fd);
> -
> -			if (size<=0) {
> -				close (fd);
> -				return -1;
> -			}
> -			dprintf("writing %d bytes\n",size);
> -			fwrite(outbuf,1, size,fout);
> -//			fflush(fout);
> -		} while (1);
> -	}
> -
> -
> -	while (1) {
> -		skip=0;
> -		header=0;
> -		do {
> -			int c;
> -			c=read_char (&drv,fd);
> -			if (c<0) {
> -				perror("read");
> -				return -1;
> -			}
> -
> -			header=(header>>8)&0xffffff;  
> -			header=header|(c<<24);
> -			skip++;
> -		} while ( (((header>>24)&0xff) != 0x47) );
> -
> -		/* split the header fields */
> -		size  = (((header & 0x7e)<<1) -1) * 4;
> -		block = (header>>7) & 0xf;
> -		field = (header>>11) & 0x1;
> -		line  = (header>>12) & 0x1ff;
> -		cmd   = (header>>21) & 0x7;
> -
> -		/* Read the remaining buffer */
> -		for (i=0;i<sizeof(buf);i++) {
> -			int c;
> -			c=read_char (&drv,fd);
> -			if (c<0) {
> -				perror("read");
> -				return -1;
> -			}
> -			buf[i]=c;
> -		}
> -
> -		/* FIXME: Mounts the image as field0+field1
> -			* It should, instead, check if the user selected
> -			* entrelaced or non-entrelaced mode
> -			*/
> -		pos=((line<<1)+field)*linesize+
> -					block*TM6000_URB_MSG_LEN;
> -
> -			/* Prints debug info */
> -		dprintf("0x%08x (skip %d), %s size=%d, line=%d, field=%d, block=%d\n",
> -				(unsigned int)header, skip,
> -				 tm6000_msg_type[cmd],
> -				 size, line, field, block);
> -
> -		/* Don't allow to write out of the buffer */
> -		if (pos+sizeof(buf) > sizeof(img))
> -			cmd = TM6000_URB_MSG_ERR;
> -
> -		/* handles each different URB message */
> -		switch(cmd) {
> -		case TM6000_URB_MSG_VIDEO:
> -			/* Fills video buffer */
> -			memcpy(buf,&img[pos],sizeof(buf));
> -		case TM6000_URB_MSG_AUDIO:
> -			if (audio)
> -				fwrite(buf,sizeof(buf),1,stdout);
> -//		case TM6000_URB_MSG_VBI:
> -//		case TM6000_URB_MSG_PTS:
> -		break;
> -		}
> -	}
> -	close(fd);
> -	return 0;
> +  v4l2_close(-2);
>  }
> 



Thanks,
Mauro

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 6866 bytes --]

Hi!

> > For focus to be useful, we need autofocus implmented
> > somewhere. Unfortunately, v4l framework does not seem to provide good
> > place where to put autofocus. I believe, long-term, we'll need some
> > kind of "video server" providing this kind of services.
> > 
> > Anyway, we probably don't want autofocus in kernel (even through some
> > cameras do it in hardware), and we probably don't want autofocus in
> > each and every user application.
> > 
> > So what remains is libv4l2. 
> 
> IMO, the best place for autofocus is at libv4l2. Putting it on a
> separate "video server" application looks really weird for me.

Well... let me see. libraries are quite limited -- it is hard to open
files, or use threads/have custom main loop. It may be useful to
switch resolutions -- do autofocus/autogain at lower resolution, then
switch to high one for taking picture. It would be good to have that
in "system" code, but I'm not at all sure libv4l2 design will allow
that.

It would be good if application could say "render live camera into
this window" and only care about user interface, then say "give me a
high resolution jpeg". But that would require main loop in the
library...

It would be nice if more than one application could be accessing the
camera at the same time... (I.e. something graphical running preview
then using command line tool to grab a picture.) This one is
definitely not solveable inside a library...

> Btw, libv4l2 already has some autotools for auto gain and auto
> white balance. See the implementation under:
> 	lib/libv4lconvert/processing
> 
> The libv4l internal controls can be seen at:
> 	lib/libv4lconvert/control/libv4lcontrol.h
> 
> The ones implemented by the processing part of the library are:

Thanks for pointer, will take a look.

> > Now, this is in no way clean or complete,
> > and functionality provided by sdl.c and asciicam.c probably _should_
> > be in application, but... I'd like to get the code out there.
> > 
> > Oh and yes, I've canibalized decode_tm6000.c application instead of
> > introducing my own. Autotools scare me, sorry.
> 
> Why replace decode_tm6000.c by something else? If you want to add another
> test application, just place it on a new file.

Scary scary scary autotools ;-). Yes, I did rather lot of hacks, as
you noted below. I do development on n900, so not everything is easy.

> I added a few notes together with the code, pointing the main things
> I think it require changes, in order for me to do a better review
> at the code. I didn't test nor tried to check the algorithms inside,
> as the code, on its current state, requires rework and code cleanup.

Thanks, I'll take a look.

> Please don't add a new application under lib/. It is fine if you want
> some testing application, if the ones there aren't enough, but please
> place it under contrib/test/.
> 
> You should likely take a look at v4l2grab first, as it could have
> almost everything you would need.

Will take a look, thanks for pointer.

> IMHO, it would be better to use aalib. Btw, xawtv3 has a code example
> using it, under:
> 	console/ttv.c
> 
> As it already uses libv4l, prhaps you could use it, instead of adding
> a new ascii app.

No need to duplicate it, then. I was trying to quickly test video
works, this was before SDL.

> > +#include "sdl.c"
> > +
> > +static struct sdl sdl;
> > +
> > +int v4l2_get_index(int fd);
> > +void my_main(void);
> > +
> 
> The above looks really odd. Why do you want to make libv4l2 dependent
> on sdl?

I don't, but I had some nasty problems with linker; this should really
go into application but it refused to link. Scary libtool.

> > +static void v4l2_histogram(unsigned char *buf, int cdf[], struct v4l2_format *fmt)
> > +{
> > +    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
> > +      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
> > +	pixel p = buf_pixel(fmt, buf, x, y);
> > +	
> > +	int b;
> > +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> > +	b = p.r + 2*p.g + p.b;
> > +	b = (b * BUCKETS)/(256);
> > +	cdf[b]++;
> > +      }
> > +}
> > +
> > +static long v4l2_sharpness(unsigned char *buf, struct v4l2_format *fmt)
> > +{
> > +  int h = fmt->fmt.pix.height;
> > +  int w = fmt->fmt.pix.width;
> > +  long r = 0;
> > +
> > +    for (int y = h/3; y < h-h/3; y+=h/9)
> > +      for (int x = w/3; x < w-w/3; x++) {
> > +	pixel p1 = buf_pixel(fmt, buf, x, y);
> > +	pixel p2 = buf_pixel(fmt, buf, x+2, y);
> > +	
> > +	int b1, b2;
> > +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> > +	b1 = p1.r + 2*p1.g + p1.b;
> > +	b2 = p2.r + 2*p2.g + p2.b;
> > +
> > +	int v;
> > +	v = (b1-b2)*(b1-b2);
> > +	if (v > 36)
> > +		r+=v;
> > +      }
> > +
> > +    return r;
> > +}
> 
> IMO, the above belongs to a separate processing module under
> 	lib/libv4lconvert/processing/

I guess so.

> > +
> > +int v4l2_set_exposure(int fd, int exposure)
> > +{
> > +	int index = v4l2_get_index(fd);
> > +
> > +	if (index == -1 || devices[index].convert == NULL) {
> > +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> > +		errno = EBADF;
> > +		return -1;
> > +	}
> > +
> > +	struct v4l2_control ctrl;
> > +	ctrl.id = V4L2_CID_EXPOSURE;
> > +	ctrl.value = exposure;
> > +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> > +	  printf("Could not set exposure\n");
> > +	}
> > +	return 0;
> > +}
> 
> Shouldn't it be together with lib/libv4lconvert/processing/autogain.c,
> perhaps as an alternative implementation, if what's there is not
> enough?

I'll take a look, thanks.

> > @@ -823,6 +1246,10 @@ int v4l2_close(int fd)
> >  {
> >  	int index, result;
> >  
> > +	if (fd == -2) {
> > +	  my_main();
> > +	}
> > +
> 
> That looks a hack!

That is _the_ hack ;-). Yes, agreed, need to look at
processing/. .. when I get time.


> > +#include <SDL2/SDL.h>
> > +#include <SDL2/SDL_image.h>
> 
> If you're adding a SDL-specific application, you'll need to add the 
> needed autoconf bits to detect if SDL devel package is installed,
> auto-disabling it if not.
> 
> Yet, I don't think that SDL should be part of the library, but,
> instead, part of some application.

Agreed. libtool prevented me from doing the right thing.

> > index 4bffbdd..fda7e3b 100644
> > --- a/utils/decode_tm6000/decode_tm6000.c
> > +++ b/utils/decode_tm6000/decode_tm6000.c
> 
> Everything below it is completely wrong!

And most of the stuff above is, too :-). I wanted to get the code out
in case I won't have time...

Thanks,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > For focus to be useful, we need autofocus implmented
> > somewhere. Unfortunately, v4l framework does not seem to provide good
> > place where to put autofocus. I believe, long-term, we'll need some
> > kind of "video server" providing this kind of services.
> > 
> > Anyway, we probably don't want autofocus in kernel (even through some
> > cameras do it in hardware), and we probably don't want autofocus in
> > each and every user application.
> > 
> > So what remains is libv4l2. 
> 
> IMO, the best place for autofocus is at libv4l2. Putting it on a
> separate "video server" application looks really weird for me.

Well... let me see. libraries are quite limited -- it is hard to open
files, or use threads/have custom main loop. It may be useful to
switch resolutions -- do autofocus/autogain at lower resolution, then
switch to high one for taking picture. It would be good to have that
in "system" code, but I'm not at all sure libv4l2 design will allow
that.

It would be good if application could say "render live camera into
this window" and only care about user interface, then say "give me a
high resolution jpeg". But that would require main loop in the
library...

It would be nice if more than one application could be accessing the
camera at the same time... (I.e. something graphical running preview
then using command line tool to grab a picture.) This one is
definitely not solveable inside a library...

> Btw, libv4l2 already has some autotools for auto gain and auto
> white balance. See the implementation under:
> 	lib/libv4lconvert/processing
> 
> The libv4l internal controls can be seen at:
> 	lib/libv4lconvert/control/libv4lcontrol.h
> 
> The ones implemented by the processing part of the library are:

Thanks for pointer, will take a look.

> > Now, this is in no way clean or complete,
> > and functionality provided by sdl.c and asciicam.c probably _should_
> > be in application, but... I'd like to get the code out there.
> > 
> > Oh and yes, I've canibalized decode_tm6000.c application instead of
> > introducing my own. Autotools scare me, sorry.
> 
> Why replace decode_tm6000.c by something else? If you want to add another
> test application, just place it on a new file.

Scary scary scary autotools ;-). Yes, I did rather lot of hacks, as
you noted below. I do development on n900, so not everything is easy.

> I added a few notes together with the code, pointing the main things
> I think it require changes, in order for me to do a better review
> at the code. I didn't test nor tried to check the algorithms inside,
> as the code, on its current state, requires rework and code cleanup.

Thanks, I'll take a look.

> Please don't add a new application under lib/. It is fine if you want
> some testing application, if the ones there aren't enough, but please
> place it under contrib/test/.
> 
> You should likely take a look at v4l2grab first, as it could have
> almost everything you would need.

Will take a look, thanks for pointer.

> IMHO, it would be better to use aalib. Btw, xawtv3 has a code example
> using it, under:
> 	console/ttv.c
> 
> As it already uses libv4l, prhaps you could use it, instead of adding
> a new ascii app.

No need to duplicate it, then. I was trying to quickly test video
works, this was before SDL.

> > +#include "sdl.c"
> > +
> > +static struct sdl sdl;
> > +
> > +int v4l2_get_index(int fd);
> > +void my_main(void);
> > +
> 
> The above looks really odd. Why do you want to make libv4l2 dependent
> on sdl?

I don't, but I had some nasty problems with linker; this should really
go into application but it refused to link. Scary libtool.

> > +static void v4l2_histogram(unsigned char *buf, int cdf[], struct v4l2_format *fmt)
> > +{
> > +    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
> > +      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
> > +	pixel p = buf_pixel(fmt, buf, x, y);
> > +	
> > +	int b;
> > +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> > +	b = p.r + 2*p.g + p.b;
> > +	b = (b * BUCKETS)/(256);
> > +	cdf[b]++;
> > +      }
> > +}
> > +
> > +static long v4l2_sharpness(unsigned char *buf, struct v4l2_format *fmt)
> > +{
> > +  int h = fmt->fmt.pix.height;
> > +  int w = fmt->fmt.pix.width;
> > +  long r = 0;
> > +
> > +    for (int y = h/3; y < h-h/3; y+=h/9)
> > +      for (int x = w/3; x < w-w/3; x++) {
> > +	pixel p1 = buf_pixel(fmt, buf, x, y);
> > +	pixel p2 = buf_pixel(fmt, buf, x+2, y);
> > +	
> > +	int b1, b2;
> > +	/* HACK: we divide green by 2 to have nice picture, undo it here. */
> > +	b1 = p1.r + 2*p1.g + p1.b;
> > +	b2 = p2.r + 2*p2.g + p2.b;
> > +
> > +	int v;
> > +	v = (b1-b2)*(b1-b2);
> > +	if (v > 36)
> > +		r+=v;
> > +      }
> > +
> > +    return r;
> > +}
> 
> IMO, the above belongs to a separate processing module under
> 	lib/libv4lconvert/processing/

I guess so.

> > +
> > +int v4l2_set_exposure(int fd, int exposure)
> > +{
> > +	int index = v4l2_get_index(fd);
> > +
> > +	if (index == -1 || devices[index].convert == NULL) {
> > +		V4L2_LOG_ERR("v4l2_set_exposure called with invalid fd: %d\n", fd);
> > +		errno = EBADF;
> > +		return -1;
> > +	}
> > +
> > +	struct v4l2_control ctrl;
> > +	ctrl.id = V4L2_CID_EXPOSURE;
> > +	ctrl.value = exposure;
> > +	if (ioctl(devices[index].subdev_fds[0], VIDIOC_S_CTRL, &ctrl) < 0) {
> > +	  printf("Could not set exposure\n");
> > +	}
> > +	return 0;
> > +}
> 
> Shouldn't it be together with lib/libv4lconvert/processing/autogain.c,
> perhaps as an alternative implementation, if what's there is not
> enough?

I'll take a look, thanks.

> > @@ -823,6 +1246,10 @@ int v4l2_close(int fd)
> >  {
> >  	int index, result;
> >  
> > +	if (fd == -2) {
> > +	  my_main();
> > +	}
> > +
> 
> That looks a hack!

That is _the_ hack ;-). Yes, agreed, need to look at
processing/. .. when I get time.


> > +#include <SDL2/SDL.h>
> > +#include <SDL2/SDL_image.h>
> 
> If you're adding a SDL-specific application, you'll need to add the 
> needed autoconf bits to detect if SDL devel package is installed,
> auto-disabling it if not.
> 
> Yet, I don't think that SDL should be part of the library, but,
> instead, part of some application.

Agreed. libtool prevented me from doing the right thing.

> > index 4bffbdd..fda7e3b 100644
> > --- a/utils/decode_tm6000/decode_tm6000.c
> > +++ b/utils/decode_tm6000/decode_tm6000.c
> 
> Everything below it is completely wrong!

And most of the stuff above is, too :-). I wanted to get the code out
in case I won't have time...

Thanks,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170424/420fafa1/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Mon, 24 Apr 2017 23:29:14 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > For focus to be useful, we need autofocus implmented
> > > somewhere. Unfortunately, v4l framework does not seem to provide good
> > > place where to put autofocus. I believe, long-term, we'll need some
> > > kind of "video server" providing this kind of services.
> > > 
> > > Anyway, we probably don't want autofocus in kernel (even through some
> > > cameras do it in hardware), and we probably don't want autofocus in
> > > each and every user application.
> > > 
> > > So what remains is libv4l2.   
> > 
> > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > separate "video server" application looks really weird for me.  
> 
> Well... let me see. libraries are quite limited -- it is hard to open
> files, or use threads/have custom main loop. It may be useful to
> switch resolutions -- do autofocus/autogain at lower resolution, then
> switch to high one for taking picture. It would be good to have that
> in "system" code, but I'm not at all sure libv4l2 design will allow
> that.

I don't see why it would be hard to open files or have threads inside
a library. There are several libraries that do that already, specially
the ones designed to be used on multimidia apps.

Resolution switch can indeed be a problem on devices that use MC
and subdev API, as a plugin would be required to teach the library
about N9 specifics (or the Kernel API should be improved to let
a generic application to better detect the hardware capabilities).

> It would be good if application could say "render live camera into
> this window" and only care about user interface, then say "give me a
> high resolution jpeg". But that would require main loop in the
> library...

Nothing prevents writing an upper layer on the top of libv4l in
order to provide such kind of functions.

> It would be nice if more than one application could be accessing the
> camera at the same time... (I.e. something graphical running preview
> then using command line tool to grab a picture.) This one is
> definitely not solveable inside a library...

Someone once suggested to have something like pulseaudio for V4L.
For such usage, a server would be interesting. Yet, I would code it
in a way that applications using libv4l will talk with such daemon
in a transparent way.

> > The above looks really odd. Why do you want to make libv4l2 dependent
> > on sdl?  
> 
> I don't, but I had some nasty problems with linker; this should really
> go into application but it refused to link. Scary libtool.

That's weird. 


> > If you're adding a SDL-specific application, you'll need to add the 
> > needed autoconf bits to detect if SDL devel package is installed,
> > auto-disabling it if not.
> > 
> > Yet, I don't think that SDL should be part of the library, but,
> > instead, part of some application.  
> 
> Agreed. libtool prevented me from doing the right thing.

if you add libSDL detection at configure.ac, you likely won't need to
deal with libtool.

On a quick look at web, it seems that there's a m4 module that does
the right thing, according with:
	https://wiki.libsdl.org/FAQLinux


Thanks,
Mauro

support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Mon, 24 Apr 2017 23:29:14 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > For focus to be useful, we need autofocus implmented
> > > somewhere. Unfortunately, v4l framework does not seem to provide good
> > > place where to put autofocus. I believe, long-term, we'll need some
> > > kind of "video server" providing this kind of services.
> > > 
> > > Anyway, we probably don't want autofocus in kernel (even through some
> > > cameras do it in hardware), and we probably don't want autofocus in
> > > each and every user application.
> > > 
> > > So what remains is libv4l2.   
> > 
> > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > separate "video server" application looks really weird for me.  
> 
> Well... let me see. libraries are quite limited -- it is hard to open
> files, or use threads/have custom main loop. It may be useful to
> switch resolutions -- do autofocus/autogain at lower resolution, then
> switch to high one for taking picture. It would be good to have that
> in "system" code, but I'm not at all sure libv4l2 design will allow
> that.

I don't see why it would be hard to open files or have threads inside
a library. There are several libraries that do that already, specially
the ones designed to be used on multimidia apps.

Resolution switch can indeed be a problem on devices that use MC
and subdev API, as a plugin would be required to teach the library
about N9 specifics (or the Kernel API should be improved to let
a generic application to better detect the hardware capabilities).

> It would be good if application could say "render live camera into
> this window" and only care about user interface, then say "give me a
> high resolution jpeg". But that would require main loop in the
> library...

Nothing prevents writing an upper layer on the top of libv4l in
order to provide such kind of functions.

> It would be nice if more than one application could be accessing the
> camera at the same time... (I.e. something graphical running preview
> then using command line tool to grab a picture.) This one is
> definitely not solveable inside a library...

Someone once suggested to have something like pulseaudio for V4L.
For such usage, a server would be interesting. Yet, I would code it
in a way that applications using libv4l will talk with such daemon
in a transparent way.

> > The above looks really odd. Why do you want to make libv4l2 dependent
> > on sdl?  
> 
> I don't, but I had some nasty problems with linker; this should really
> go into application but it refused to link. Scary libtool.

That's weird. 


> > If you're adding a SDL-specific application, you'll need to add the 
> > needed autoconf bits to detect if SDL devel package is installed,
> > auto-disabling it if not.
> > 
> > Yet, I don't think that SDL should be part of the library, but,
> > instead, part of some application.  
> 
> Agreed. libtool prevented me from doing the right thing.

if you add libSDL detection at configure.ac, you likely won't need to
deal with libtool.

On a quick look at web, it seems that there's a m4 module that does
the right thing, according with:
	https://wiki.libsdl.org/FAQLinux


Thanks,
Mauro

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2684 bytes --]

Hi!

> > > > For focus to be useful, we need autofocus implmented
> > > > somewhere. Unfortunately, v4l framework does not seem to provide good
> > > > place where to put autofocus. I believe, long-term, we'll need some
> > > > kind of "video server" providing this kind of services.
> > > > 
> > > > Anyway, we probably don't want autofocus in kernel (even through some
> > > > cameras do it in hardware), and we probably don't want autofocus in
> > > > each and every user application.
> > > > 
> > > > So what remains is libv4l2.   
> > > 
> > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > separate "video server" application looks really weird for me.  
> > 
> > Well... let me see. libraries are quite limited -- it is hard to open
> > files, or use threads/have custom main loop. It may be useful to
> > switch resolutions -- do autofocus/autogain at lower resolution, then
> > switch to high one for taking picture. It would be good to have that
> > in "system" code, but I'm not at all sure libv4l2 design will allow
> > that.
> 
> I don't see why it would be hard to open files or have threads inside
> a library. There are several libraries that do that already, specially
> the ones designed to be used on multimidia apps.

Well, fd's are hard, because application can do fork() and now
interesting stuff happens. Threads are tricky, because now you have
locking etc.

libv4l2 is designed to be LD_PRELOADED. That is not really feasible
with "complex" library.

> > It would be good if application could say "render live camera into
> > this window" and only care about user interface, then say "give me a
> > high resolution jpeg". But that would require main loop in the
> > library...
> 
> Nothing prevents writing an upper layer on the top of libv4l in
> order to provide such kind of functions.

Agreed.

> > It would be nice if more than one application could be accessing the
> > camera at the same time... (I.e. something graphical running preview
> > then using command line tool to grab a picture.) This one is
> > definitely not solveable inside a library...
> 
> Someone once suggested to have something like pulseaudio for V4L.
> For such usage, a server would be interesting. Yet, I would code it
> in a way that applications using libv4l will talk with such daemon
> in a transparent way.

Yes, we need something like pulseaudio for V4L. And yes, we should
make it transparent for applications using libv4l.

Regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > > > For focus to be useful, we need autofocus implmented
> > > > somewhere. Unfortunately, v4l framework does not seem to provide good
> > > > place where to put autofocus. I believe, long-term, we'll need some
> > > > kind of "video server" providing this kind of services.
> > > > 
> > > > Anyway, we probably don't want autofocus in kernel (even through some
> > > > cameras do it in hardware), and we probably don't want autofocus in
> > > > each and every user application.
> > > > 
> > > > So what remains is libv4l2.   
> > > 
> > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > separate "video server" application looks really weird for me.  
> > 
> > Well... let me see. libraries are quite limited -- it is hard to open
> > files, or use threads/have custom main loop. It may be useful to
> > switch resolutions -- do autofocus/autogain at lower resolution, then
> > switch to high one for taking picture. It would be good to have that
> > in "system" code, but I'm not at all sure libv4l2 design will allow
> > that.
> 
> I don't see why it would be hard to open files or have threads inside
> a library. There are several libraries that do that already, specially
> the ones designed to be used on multimidia apps.

Well, fd's are hard, because application can do fork() and now
interesting stuff happens. Threads are tricky, because now you have
locking etc.

libv4l2 is designed to be LD_PRELOADED. That is not really feasible
with "complex" library.

> > It would be good if application could say "render live camera into
> > this window" and only care about user interface, then say "give me a
> > high resolution jpeg". But that would require main loop in the
> > library...
> 
> Nothing prevents writing an upper layer on the top of libv4l in
> order to provide such kind of functions.

Agreed.

> > It would be nice if more than one application could be accessing the
> > camera at the same time... (I.e. something graphical running preview
> > then using command line tool to grab a picture.) This one is
> > definitely not solveable inside a library...
> 
> Someone once suggested to have something like pulseaudio for V4L.
> For such usage, a server would be interesting. Yet, I would code it
> in a way that applications using libv4l will talk with such daemon
> in a transparent way.

Yes, we need something like pulseaudio for V4L. And yes, we should
make it transparent for applications using libv4l.

Regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/576eccb4/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > It would be nice if more than one application could be accessing the
> > > camera at the same time... (I.e. something graphical running preview
> > > then using command line tool to grab a picture.) This one is
> > > definitely not solveable inside a library...
> > 
> > Someone once suggested to have something like pulseaudio for V4L.
> > For such usage, a server would be interesting. Yet, I would code it
> > in a way that applications using libv4l will talk with such daemon
> > in a transparent way.
> 
> Yes, we need something like pulseaudio for V4L. And yes, we should
> make it transparent for applications using libv4l.

IIRC there is already some effort in writing such "video" server which
would support accessing more application into webcam video, like
pulseaudio server for accessing more applications to microphone input.

-- 
Pali Rohár
pali.rohar@gmail.com

support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > It would be nice if more than one application could be accessing the
> > > camera at the same time... (I.e. something graphical running preview
> > > then using command line tool to grab a picture.) This one is
> > > definitely not solveable inside a library...
> > 
> > Someone once suggested to have something like pulseaudio for V4L.
> > For such usage, a server would be interesting. Yet, I would code it
> > in a way that applications using libv4l will talk with such daemon
> > in a transparent way.
> 
> Yes, we need something like pulseaudio for V4L. And yes, we should
> make it transparent for applications using libv4l.

IIRC there is already some effort in writing such "video" server which
would support accessing more application into webcam video, like
pulseaudio server for accessing more applications to microphone input.

-- 
Pali Roh?r
pali.rohar at gmail.com

Re: support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > It would be nice if more than one application could be accessing the
> > > camera at the same time... (I.e. something graphical running preview
> > > then using command line tool to grab a picture.) This one is
> > > definitely not solveable inside a library...
> > 
> > Someone once suggested to have something like pulseaudio for V4L.
> > For such usage, a server would be interesting. Yet, I would code it
> > in a way that applications using libv4l will talk with such daemon
> > in a transparent way.
> 
> Yes, we need something like pulseaudio for V4L. And yes, we should
> make it transparent for applications using libv4l.

IIRC there is already some effort in writing such "video" server which
would support accessing more application into webcam video, like
pulseaudio server for accessing more applications to microphone input.

-- 
Pali Rohár
pali.rohar@gmail.com

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1215 bytes --]

Hi!
On Tue 2017-04-25 10:08:15, Pali Rohár wrote:
> On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > It would be nice if more than one application could be accessing the
> > > > camera at the same time... (I.e. something graphical running preview
> > > > then using command line tool to grab a picture.) This one is
> > > > definitely not solveable inside a library...
> > > 
> > > Someone once suggested to have something like pulseaudio for V4L.
> > > For such usage, a server would be interesting. Yet, I would code it
> > > in a way that applications using libv4l will talk with such daemon
> > > in a transparent way.
> > 
> > Yes, we need something like pulseaudio for V4L. And yes, we should
> > make it transparent for applications using libv4l.
> 
> IIRC there is already some effort in writing such "video" server which
> would support accessing more application into webcam video, like
> pulseaudio server for accessing more applications to microphone input.

Do you have project name / url / something?

Thanks,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!
On Tue 2017-04-25 10:08:15, Pali Roh?r wrote:
> On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > It would be nice if more than one application could be accessing the
> > > > camera at the same time... (I.e. something graphical running preview
> > > > then using command line tool to grab a picture.) This one is
> > > > definitely not solveable inside a library...
> > > 
> > > Someone once suggested to have something like pulseaudio for V4L.
> > > For such usage, a server would be interesting. Yet, I would code it
> > > in a way that applications using libv4l will talk with such daemon
> > > in a transparent way.
> > 
> > Yes, we need something like pulseaudio for V4L. And yes, we should
> > make it transparent for applications using libv4l.
> 
> IIRC there is already some effort in writing such "video" server which
> would support accessing more application into webcam video, like
> pulseaudio server for accessing more applications to microphone input.

Do you have project name / url / something?

Thanks,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/b6562bbd/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> Hi!
> On Tue 2017-04-25 10:08:15, Pali Rohár wrote:
> > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > It would be nice if more than one application could be accessing the
> > > > > camera at the same time... (I.e. something graphical running preview
> > > > > then using command line tool to grab a picture.) This one is
> > > > > definitely not solveable inside a library...
> > > > 
> > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > For such usage, a server would be interesting. Yet, I would code it
> > > > in a way that applications using libv4l will talk with such daemon
> > > > in a transparent way.
> > > 
> > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > make it transparent for applications using libv4l.
> > 
> > IIRC there is already some effort in writing such "video" server which
> > would support accessing more application into webcam video, like
> > pulseaudio server for accessing more applications to microphone input.
> 
> Do you have project name / url / something?

Pinos (renamed from PulseVideo)

https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
https://cgit.freedesktop.org/~wtay/pinos/

But from git history it looks like it is probably dead now...

-- 
Pali Rohár
pali.rohar@gmail.com

support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> Hi!
> On Tue 2017-04-25 10:08:15, Pali Roh?r wrote:
> > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > It would be nice if more than one application could be accessing the
> > > > > camera at the same time... (I.e. something graphical running preview
> > > > > then using command line tool to grab a picture.) This one is
> > > > > definitely not solveable inside a library...
> > > > 
> > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > For such usage, a server would be interesting. Yet, I would code it
> > > > in a way that applications using libv4l will talk with such daemon
> > > > in a transparent way.
> > > 
> > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > make it transparent for applications using libv4l.
> > 
> > IIRC there is already some effort in writing such "video" server which
> > would support accessing more application into webcam video, like
> > pulseaudio server for accessing more applications to microphone input.
> 
> Do you have project name / url / something?

Pinos (renamed from PulseVideo)

https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
https://cgit.freedesktop.org/~wtay/pinos/

But from git history it looks like it is probably dead now...

-- 
Pali Roh?r
pali.rohar at gmail.com

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1726 bytes --]

On Tue 2017-04-25 13:30:09, Pali Rohár wrote:
> On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> > Hi!
> > On Tue 2017-04-25 10:08:15, Pali Rohár wrote:
> > > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > > It would be nice if more than one application could be accessing the
> > > > > > camera at the same time... (I.e. something graphical running preview
> > > > > > then using command line tool to grab a picture.) This one is
> > > > > > definitely not solveable inside a library...
> > > > > 
> > > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > > For such usage, a server would be interesting. Yet, I would code it
> > > > > in a way that applications using libv4l will talk with such daemon
> > > > > in a transparent way.
> > > > 
> > > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > > make it transparent for applications using libv4l.
> > > 
> > > IIRC there is already some effort in writing such "video" server which
> > > would support accessing more application into webcam video, like
> > > pulseaudio server for accessing more applications to microphone input.
> > 
> > Do you have project name / url / something?
> 
> Pinos (renamed from PulseVideo)
> 
> https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> https://cgit.freedesktop.org/~wtay/pinos/
> 
> But from git history it looks like it is probably dead now...

Actually, last commit is an hour ago on "work" branch. Seems alive to
me ;-).

Thanks for pointer...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

On Tue 2017-04-25 13:30:09, Pali Roh?r wrote:
> On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> > Hi!
> > On Tue 2017-04-25 10:08:15, Pali Roh?r wrote:
> > > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > > It would be nice if more than one application could be accessing the
> > > > > > camera at the same time... (I.e. something graphical running preview
> > > > > > then using command line tool to grab a picture.) This one is
> > > > > > definitely not solveable inside a library...
> > > > > 
> > > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > > For such usage, a server would be interesting. Yet, I would code it
> > > > > in a way that applications using libv4l will talk with such daemon
> > > > > in a transparent way.
> > > > 
> > > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > > make it transparent for applications using libv4l.
> > > 
> > > IIRC there is already some effort in writing such "video" server which
> > > would support accessing more application into webcam video, like
> > > pulseaudio server for accessing more applications to microphone input.
> > 
> > Do you have project name / url / something?
> 
> Pinos (renamed from PulseVideo)
> 
> https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> https://cgit.freedesktop.org/~wtay/pinos/
> 
> But from git history it looks like it is probably dead now...

Actually, last commit is an hour ago on "work" branch. Seems alive to
me ;-).

Thanks for pointer...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/cdcc4df0/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 14:28:20 Pavel Machek wrote:
> On Tue 2017-04-25 13:30:09, Pali Rohár wrote:
> > On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> > > Hi!
> > > On Tue 2017-04-25 10:08:15, Pali Rohár wrote:
> > > > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > > > It would be nice if more than one application could be accessing the
> > > > > > > camera at the same time... (I.e. something graphical running preview
> > > > > > > then using command line tool to grab a picture.) This one is
> > > > > > > definitely not solveable inside a library...
> > > > > > 
> > > > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > > > For such usage, a server would be interesting. Yet, I would code it
> > > > > > in a way that applications using libv4l will talk with such daemon
> > > > > > in a transparent way.
> > > > > 
> > > > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > > > make it transparent for applications using libv4l.
> > > > 
> > > > IIRC there is already some effort in writing such "video" server which
> > > > would support accessing more application into webcam video, like
> > > > pulseaudio server for accessing more applications to microphone input.
> > > 
> > > Do you have project name / url / something?
> > 
> > Pinos (renamed from PulseVideo)
> > 
> > https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> > https://cgit.freedesktop.org/~wtay/pinos/
> > 
> > But from git history it looks like it is probably dead now...
> 
> Actually, last commit is an hour ago on "work" branch. Seems alive to
> me ;-).

Great! I just (blindly) looked at master branch and it is old...

> Thanks for pointer...
> 									Pavel

-- 
Pali Rohár
pali.rohar@gmail.com

support autofocus / autogain in libv4l2

[Pali Rohár]

On Tuesday 25 April 2017 14:28:20 Pavel Machek wrote:
> On Tue 2017-04-25 13:30:09, Pali Roh?r wrote:
> > On Tuesday 25 April 2017 13:23:30 Pavel Machek wrote:
> > > Hi!
> > > On Tue 2017-04-25 10:08:15, Pali Roh?r wrote:
> > > > On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > > > > It would be nice if more than one application could be accessing the
> > > > > > > camera at the same time... (I.e. something graphical running preview
> > > > > > > then using command line tool to grab a picture.) This one is
> > > > > > > definitely not solveable inside a library...
> > > > > > 
> > > > > > Someone once suggested to have something like pulseaudio for V4L.
> > > > > > For such usage, a server would be interesting. Yet, I would code it
> > > > > > in a way that applications using libv4l will talk with such daemon
> > > > > > in a transparent way.
> > > > > 
> > > > > Yes, we need something like pulseaudio for V4L. And yes, we should
> > > > > make it transparent for applications using libv4l.
> > > > 
> > > > IIRC there is already some effort in writing such "video" server which
> > > > would support accessing more application into webcam video, like
> > > > pulseaudio server for accessing more applications to microphone input.
> > > 
> > > Do you have project name / url / something?
> > 
> > Pinos (renamed from PulseVideo)
> > 
> > https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> > https://cgit.freedesktop.org/~wtay/pinos/
> > 
> > But from git history it looks like it is probably dead now...
> 
> Actually, last commit is an hour ago on "work" branch. Seems alive to
> me ;-).

Great! I just (blindly) looked at master branch and it is old...

> Thanks for pointer...
> 									Pavel

-- 
Pali Roh?r
pali.rohar at gmail.com

Re: support autofocus / autogain in libv4l2

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 484 bytes --]

Le mardi 25 avril 2017 à 13:30 +0200, Pali Rohár a écrit :
> Pinos (renamed from PulseVideo)
> 
> https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> https://cgit.freedesktop.org/~wtay/pinos/
> 
> But from git history it looks like it is probably dead now...

This is also incorrect. See "work" branch. It is still a one man show,
code being aggressively re-factored. I suspect this will be the case
until the "form" is considered acceptable.

Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Nicolas Dufresne]

Le mardi 25 avril 2017 ? 13:30 +0200, Pali Roh?r a ?crit?:
> Pinos (renamed from PulseVideo)
> 
> https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
> https://cgit.freedesktop.org/~wtay/pinos/
> 
> But from git history it looks like it is probably dead now...

This is also incorrect. See "work" branch. It is still a one man show,
code being aggressively re-factored. I suspect this will be the case
until the "form" is considered acceptable.

Nicolas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/7a9df95e/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 1429 bytes --]

Le mardi 25 avril 2017 à 10:08 +0200, Pali Rohár a écrit :
> On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > It would be nice if more than one application could be
> > > > accessing the
> > > > camera at the same time... (I.e. something graphical running
> > > > preview
> > > > then using command line tool to grab a picture.) This one is
> > > > definitely not solveable inside a library...
> > > 
> > > Someone once suggested to have something like pulseaudio for V4L.
> > > For such usage, a server would be interesting. Yet, I would code
> > > it
> > > in a way that applications using libv4l will talk with such
> > > daemon
> > > in a transparent way.
> > 
> > Yes, we need something like pulseaudio for V4L. And yes, we should
> > make it transparent for applications using libv4l.
> 
> IIRC there is already some effort in writing such "video" server
> which
> would support accessing more application into webcam video, like
> pulseaudio server for accessing more applications to microphone
> input.
> 

Because references are nice:

https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
https://gstconf.ubicast.tv/videos/camera-sharing-and-sandboxing-with-pinos/

And why the internals are not going to be implemented using GStreamer in the end:
https://gstconf.ubicast.tv/videos/keep-calm-and-refactor-about-the-essence-of-gstreamer/

regards,
Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Nicolas Dufresne]

Le mardi 25 avril 2017 ? 10:08 +0200, Pali Roh?r a ?crit?:
> On Tuesday 25 April 2017 10:05:38 Pavel Machek wrote:
> > > > It would be nice if more than one application could be
> > > > accessing the
> > > > camera at the same time... (I.e. something graphical running
> > > > preview
> > > > then using command line tool to grab a picture.) This one is
> > > > definitely not solveable inside a library...
> > > 
> > > Someone once suggested to have something like pulseaudio for V4L.
> > > For such usage, a server would be interesting. Yet, I would code
> > > it
> > > in a way that applications using libv4l will talk with such
> > > daemon
> > > in a transparent way.
> > 
> > Yes, we need something like pulseaudio for V4L. And yes, we should
> > make it transparent for applications using libv4l.
> 
> IIRC there is already some effort in writing such "video" server
> which
> would support accessing more application into webcam video, like
> pulseaudio server for accessing more applications to microphone
> input.
> 

Because references are nice:

https://blogs.gnome.org/uraeus/2015/06/30/introducing-pulse-video/
https://gstconf.ubicast.tv/videos/camera-sharing-and-sandboxing-with-pinos/

And why the internals are not going to be implemented using GStreamer in the end:
https://gstconf.ubicast.tv/videos/keep-calm-and-refactor-about-the-essence-of-gstreamer/

regards,
Nicolas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/c6b3f95c/attachment-0001.sig>

Re: support autofocus / autogain in libv4l2

[Nicolas Dufresne]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Le mardi 25 avril 2017 à 10:05 +0200, Pavel Machek a écrit :
> Well, fd's are hard, because application can do fork() and now
> interesting stuff happens. Threads are tricky, because now you have
> locking etc.
> 
> libv4l2 is designed to be LD_PRELOADED. That is not really feasible
> with "complex" library.

That is incorrect. The library propose an API where you simply replace
certain low level calls, like ioctl -> v4l2_ioctl, open -> v4l2_open().
You have to do that explicitly in your existing code. It does not
abstract the API itself unlike libdrm.

Nicolas

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Nicolas Dufresne]

Le mardi 25 avril 2017 ? 10:05 +0200, Pavel Machek a ?crit?:
> Well, fd's are hard, because application can do fork() and now
> interesting stuff happens. Threads are tricky, because now you have
> locking etc.
> 
> libv4l2 is designed to be LD_PRELOADED. That is not really feasible
> with "complex" library.

That is incorrect. The library propose an API where you simply replace
certain low level calls, like ioctl -> v4l2_ioctl, open -> v4l2_open().
You have to do that explicitly in your existing code. It does not
abstract the API itself unlike libdrm.

Nicolas
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: This is a digitally signed message part
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/82d4ed25/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Nicolas Dufresne]

[-- Attachment #1.1: Type: text/plain, Size: 583 bytes --]

Le mardi 25 avril 2017 à 10:05 +0200, Pavel Machek a écrit :
> Well, fd's are hard, because application can do fork() and now
> interesting stuff happens. Threads are tricky, because now you have
> locking etc.
> 
> libv4l2 is designed to be LD_PRELOADED. That is not really feasible
> with "complex" library.

That is incorrect. The library propose an API where you simply replace
certain low level calls, like ioctl -> v4l2_ioctl, open -> v4l2_open().
You have to do that explicitly in your existing code. It does not
abstract the API itself unlike libdrm.

Nicolas

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 912 bytes --]

On Tue 2017-04-25 12:53:27, Nicolas Dufresne wrote:
> Le mardi 25 avril 2017 à 10:05 +0200, Pavel Machek a écrit :
> > Well, fd's are hard, because application can do fork() and now
> > interesting stuff happens. Threads are tricky, because now you have
> > locking etc.
> > 
> > libv4l2 is designed to be LD_PRELOADED. That is not really feasible
> > with "complex" library.
> 
> That is incorrect. The library propose an API where you simply replace
> certain low level calls, like ioctl -> v4l2_ioctl, open -> v4l2_open().
> You have to do that explicitly in your existing code. It does not
> abstract the API itself unlike libdrm.

You are right, no LD_PRELOAD. But same API as kernel, which is really
limiting -- see my other mail.

									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

On Tue 2017-04-25 12:53:27, Nicolas Dufresne wrote:
> Le mardi 25 avril 2017 ? 10:05 +0200, Pavel Machek a ?crit?:
> > Well, fd's are hard, because application can do fork() and now
> > interesting stuff happens. Threads are tricky, because now you have
> > locking etc.
> > 
> > libv4l2 is designed to be LD_PRELOADED. That is not really feasible
> > with "complex" library.
> 
> That is incorrect. The library propose an API where you simply replace
> certain low level calls, like ioctl -> v4l2_ioctl, open -> v4l2_open().
> You have to do that explicitly in your existing code. It does not
> abstract the API itself unlike libdrm.

You are right, no LD_PRELOAD. But same API as kernel, which is really
limiting -- see my other mail.

									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170426/022f73cb/attachment.sig>

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 5207 bytes --]

Hi!

> > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > separate "video server" application looks really weird for me.  
> > 
> > Well... let me see. libraries are quite limited -- it is hard to open
> > files, or use threads/have custom main loop. It may be useful to
> > switch resolutions -- do autofocus/autogain at lower resolution, then
> > switch to high one for taking picture. It would be good to have that
> > in "system" code, but I'm not at all sure libv4l2 design will allow
> > that.
> 
> I don't see why it would be hard to open files or have threads inside
> a library. There are several libraries that do that already, specially
> the ones designed to be used on multimidia apps.

Well, This is what the libv4l2 says:

   This file implements libv4l2, which offers v4l2_ prefixed versions
   of
      open/close/etc. The API is 100% the same as directly opening
   /dev/videoX
      using regular open/close/etc, the big difference is that format
   conversion
   
but if I open additional files in v4l2_open(), API is no longer the
same, as unix open() is defined to open just one file descriptor.

Now. There is autogain support in libv4lconvert, but it expects to use
same fd for camera and for the gain... which does not work with
subdevs.

Of course, opening subdevs by name like this is not really
acceptable. But can you suggest a method that is?

Thanks,
								Pavel

commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
Author: Pavel <pavel@ucw.cz>
Date:   Wed Apr 26 11:38:04 2017 +0200

    Add subdevices.

diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 343db5e..a6bc48e 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -26,6 +26,7 @@
 #include "../libv4lconvert/libv4lsyscall-priv.h"
 
 #define V4L2_MAX_DEVICES 16
+#define V4L2_MAX_SUBDEVS 8
 /* Warning when making this larger the frame_queued and frame_mapped members of
    the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
    be adjusted! */
@@ -104,6 +105,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+        int subdev_fds[V4L2_MAX_SUBDEVS];
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 0ba0a88..edc9642 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
 
@@ -789,18 +790,25 @@ no_capture:
 
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
-	   a fram erate using the S_PARM ioctl after a S_FMT */
+	   a frame rate using the S_PARM ioctl after a S_FMT */
 	if (devices[index].convert)
 		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
 	v4l2_update_fps(index, &parm);
 
+	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
+	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
+	devices[index].subdev_fds[2] = -1;
+
+	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
+	       devices[index].subdev_fds[1]);
+
 	V4L2_LOG("open: %d\n", fd);
 
 	return fd;
 }
 
 /* Is this an fd for which we are emulating v4l1 ? */
-static int v4l2_get_index(int fd)
+int v4l2_get_index(int fd)
 {
 	int index;
 

commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
Author: Pavel <pavel@ucw.cz>
Date:   Wed Apr 26 12:34:04 2017 +0200

    Enable ioctl propagation.

diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index edc9642..6dab661 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
+{
+	int i = 0;
+	int result;
+	while (1) {
+		if (devices[index].subdev_fds[i] == -1)
+			return -1;
+		printf("g_ctrl failed, trying...\n");
+		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
+		printf("subdev %d result %d\n", i, result);
+		if (result == 0)
+			return 0;
+		i++;
+	}
+	return -1;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
@@ -1193,14 +1210,20 @@ no_capture_request:
 	switch (request) {
 	case VIDIOC_QUERYCTRL:
 		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_CTRL:
 		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_S_CTRL:
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_EXT_CTRLS:


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > separate "video server" application looks really weird for me.  
> > 
> > Well... let me see. libraries are quite limited -- it is hard to open
> > files, or use threads/have custom main loop. It may be useful to
> > switch resolutions -- do autofocus/autogain at lower resolution, then
> > switch to high one for taking picture. It would be good to have that
> > in "system" code, but I'm not at all sure libv4l2 design will allow
> > that.
> 
> I don't see why it would be hard to open files or have threads inside
> a library. There are several libraries that do that already, specially
> the ones designed to be used on multimidia apps.

Well, This is what the libv4l2 says:

   This file implements libv4l2, which offers v4l2_ prefixed versions
   of
      open/close/etc. The API is 100% the same as directly opening
   /dev/videoX
      using regular open/close/etc, the big difference is that format
   conversion
   
but if I open additional files in v4l2_open(), API is no longer the
same, as unix open() is defined to open just one file descriptor.

Now. There is autogain support in libv4lconvert, but it expects to use
same fd for camera and for the gain... which does not work with
subdevs.

Of course, opening subdevs by name like this is not really
acceptable. But can you suggest a method that is?

Thanks,
								Pavel

commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
Author: Pavel <pavel@ucw.cz>
Date:   Wed Apr 26 11:38:04 2017 +0200

    Add subdevices.

diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 343db5e..a6bc48e 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -26,6 +26,7 @@
 #include "../libv4lconvert/libv4lsyscall-priv.h"
 
 #define V4L2_MAX_DEVICES 16
+#define V4L2_MAX_SUBDEVS 8
 /* Warning when making this larger the frame_queued and frame_mapped members of
    the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
    be adjusted! */
@@ -104,6 +105,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+        int subdev_fds[V4L2_MAX_SUBDEVS];
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 0ba0a88..edc9642 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
 
@@ -789,18 +790,25 @@ no_capture:
 
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
-	   a fram erate using the S_PARM ioctl after a S_FMT */
+	   a frame rate using the S_PARM ioctl after a S_FMT */
 	if (devices[index].convert)
 		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
 	v4l2_update_fps(index, &parm);
 
+	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
+	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
+	devices[index].subdev_fds[2] = -1;
+
+	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
+	       devices[index].subdev_fds[1]);
+
 	V4L2_LOG("open: %d\n", fd);
 
 	return fd;
 }
 
 /* Is this an fd for which we are emulating v4l1 ? */
-static int v4l2_get_index(int fd)
+int v4l2_get_index(int fd)
 {
 	int index;
 

commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
Author: Pavel <pavel@ucw.cz>
Date:   Wed Apr 26 12:34:04 2017 +0200

    Enable ioctl propagation.

diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index edc9642..6dab661 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
+{
+	int i = 0;
+	int result;
+	while (1) {
+		if (devices[index].subdev_fds[i] == -1)
+			return -1;
+		printf("g_ctrl failed, trying...\n");
+		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
+		printf("subdev %d result %d\n", i, result);
+		if (result == 0)
+			return 0;
+		i++;
+	}
+	return -1;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
@@ -1193,14 +1210,20 @@ no_capture_request:
 	switch (request) {
 	case VIDIOC_QUERYCTRL:
 		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_CTRL:
 		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_S_CTRL:
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_EXT_CTRLS:


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170426/d174a739/attachment-0001.sig>

Re: [patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Hi Pavel,

Em Wed, 26 Apr 2017 12:53:00 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > > separate "video server" application looks really weird for me.    
> > > 
> > > Well... let me see. libraries are quite limited -- it is hard to open
> > > files, or use threads/have custom main loop. It may be useful to
> > > switch resolutions -- do autofocus/autogain at lower resolution, then
> > > switch to high one for taking picture. It would be good to have that
> > > in "system" code, but I'm not at all sure libv4l2 design will allow
> > > that.  
> > 
> > I don't see why it would be hard to open files or have threads inside
> > a library. There are several libraries that do that already, specially
> > the ones designed to be used on multimidia apps.  
> 
> Well, This is what the libv4l2 says:
> 
>    This file implements libv4l2, which offers v4l2_ prefixed versions
>    of
>       open/close/etc. The API is 100% the same as directly opening
>    /dev/videoX
>       using regular open/close/etc, the big difference is that format
>    conversion
>    
> but if I open additional files in v4l2_open(), API is no longer the
> same, as unix open() is defined to open just one file descriptor.
> 
> Now. There is autogain support in libv4lconvert, but it expects to use
> same fd for camera and for the gain... which does not work with
> subdevs.
> 
> Of course, opening subdevs by name like this is not really
> acceptable. But can you suggest a method that is?

There are two separate things here:

1) Autofoucs for a device that doesn't use subdev API
2) libv4l2 support for devices that require MC and subdev API

for (1), it should use the /dev/videoX device that was opened with
v4l2_open().

For (2), libv4l2 should be aware of MC and subdev APIs. Sakari
once tried to write a libv4l2 plugin for OMAP3, but never finished it.
A more recent trial were to add a libv4l2 plugin for Exynos.
Unfortunately, none of those code got merged. Last time I checked,
the Exynos plugin was almost ready to be merged, but Sakari asked
some changes on it. The developer that was working on it got job on
some other company. Last time I heard from him, he was still interested
on finishing his work, but in the need to setup a test environment
using his own devices.

So, currently, there's no code at all adding MC/subdev API
support merged at libv4l2.

-

IMHO, the right thing to do with regards to autofocus is to
implement it via a processing module, assuming that just one
video device is opened.

Then, add a N900 plugin to make libv4l2 aware of OMAP3 specifics.

After that, rework at the processing module to let it use a 
different file descriptor if such plugin is in usage.

-

The hole idea is that a libv4l2 client, running on a N900 device
would just open a fake /dev/video0. Internally, libv4l2 will
open whatever video nodes it needs to control the device, exporting
all hardware capabilities (video formats, controls, resolutions,
etc) as if it was a normal V4L2 camera, hiding all dirty details
about MC and subdev APIs from userspace application.

This way, a normal application, like xawtv, tvtime, camorama,
zbar, mplayer, vlc, ... will work without any changes.


> 
> Thanks,
> 								Pavel
> 
> commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 11:38:04 2017 +0200
> 
>     Add subdevices.
> 
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..a6bc48e 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -26,6 +26,7 @@
>  #include "../libv4lconvert/libv4lsyscall-priv.h"
>  
>  #define V4L2_MAX_DEVICES 16
> +#define V4L2_MAX_SUBDEVS 8
>  /* Warning when making this larger the frame_queued and frame_mapped members of
>     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
>     be adjusted! */
> @@ -104,6 +105,7 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..edc9642 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */
>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -789,18 +790,25 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +	devices[index].subdev_fds[2] = -1;
> +
> +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> +	       devices[index].subdev_fds[1]);
> +
>  	V4L2_LOG("open: %d\n", fd);
>  
>  	return fd;
>  }
>  
>  /* Is this an fd for which we are emulating v4l1 ? */
> -static int v4l2_get_index(int fd)
> +int v4l2_get_index(int fd)
>  {
>  	int index;
>  
> 
> commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 12:34:04 2017 +0200
> 
>     Enable ioctl propagation.
> 
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index edc9642..6dab661 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
>  	return 0;
>  }
>  
> +static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
> +{
> +	int i = 0;
> +	int result;
> +	while (1) {
> +		if (devices[index].subdev_fds[i] == -1)
> +			return -1;
> +		printf("g_ctrl failed, trying...\n");
> +		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
> +		printf("subdev %d result %d\n", i, result);
> +		if (result == 0)
> +			return 0;
> +		i++;
> +	}
> +	return -1;
> +}
> +
>  int v4l2_ioctl(int fd, unsigned long int request, ...)
>  {
>  	void *arg;
> @@ -1193,14 +1210,20 @@ no_capture_request:
>  	switch (request) {
>  	case VIDIOC_QUERYCTRL:
>  		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_G_CTRL:
>  		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_S_CTRL:
>  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_G_EXT_CTRLS:
> 
> 



Thanks,
Mauro

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Hi Pavel,

Em Wed, 26 Apr 2017 12:53:00 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > > separate "video server" application looks really weird for me.    
> > > 
> > > Well... let me see. libraries are quite limited -- it is hard to open
> > > files, or use threads/have custom main loop. It may be useful to
> > > switch resolutions -- do autofocus/autogain at lower resolution, then
> > > switch to high one for taking picture. It would be good to have that
> > > in "system" code, but I'm not at all sure libv4l2 design will allow
> > > that.  
> > 
> > I don't see why it would be hard to open files or have threads inside
> > a library. There are several libraries that do that already, specially
> > the ones designed to be used on multimidia apps.  
> 
> Well, This is what the libv4l2 says:
> 
>    This file implements libv4l2, which offers v4l2_ prefixed versions
>    of
>       open/close/etc. The API is 100% the same as directly opening
>    /dev/videoX
>       using regular open/close/etc, the big difference is that format
>    conversion
>    
> but if I open additional files in v4l2_open(), API is no longer the
> same, as unix open() is defined to open just one file descriptor.
> 
> Now. There is autogain support in libv4lconvert, but it expects to use
> same fd for camera and for the gain... which does not work with
> subdevs.
> 
> Of course, opening subdevs by name like this is not really
> acceptable. But can you suggest a method that is?

There are two separate things here:

1) Autofoucs for a device that doesn't use subdev API
2) libv4l2 support for devices that require MC and subdev API

for (1), it should use the /dev/videoX device that was opened with
v4l2_open().

For (2), libv4l2 should be aware of MC and subdev APIs. Sakari
once tried to write a libv4l2 plugin for OMAP3, but never finished it.
A more recent trial were to add a libv4l2 plugin for Exynos.
Unfortunately, none of those code got merged. Last time I checked,
the Exynos plugin was almost ready to be merged, but Sakari asked
some changes on it. The developer that was working on it got job on
some other company. Last time I heard from him, he was still interested
on finishing his work, but in the need to setup a test environment
using his own devices.

So, currently, there's no code at all adding MC/subdev API
support merged at libv4l2.

-

IMHO, the right thing to do with regards to autofocus is to
implement it via a processing module, assuming that just one
video device is opened.

Then, add a N900 plugin to make libv4l2 aware of OMAP3 specifics.

After that, rework at the processing module to let it use a 
different file descriptor if such plugin is in usage.

-

The hole idea is that a libv4l2 client, running on a N900 device
would just open a fake /dev/video0. Internally, libv4l2 will
open whatever video nodes it needs to control the device, exporting
all hardware capabilities (video formats, controls, resolutions,
etc) as if it was a normal V4L2 camera, hiding all dirty details
about MC and subdev APIs from userspace application.

This way, a normal application, like xawtv, tvtime, camorama,
zbar, mplayer, vlc, ... will work without any changes.


> 
> Thanks,
> 								Pavel
> 
> commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 11:38:04 2017 +0200
> 
>     Add subdevices.
> 
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..a6bc48e 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -26,6 +26,7 @@
>  #include "../libv4lconvert/libv4lsyscall-priv.h"
>  
>  #define V4L2_MAX_DEVICES 16
> +#define V4L2_MAX_SUBDEVS 8
>  /* Warning when making this larger the frame_queued and frame_mapped members of
>     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
>     be adjusted! */
> @@ -104,6 +105,7 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..edc9642 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */
>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -789,18 +790,25 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +	devices[index].subdev_fds[2] = -1;
> +
> +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> +	       devices[index].subdev_fds[1]);
> +
>  	V4L2_LOG("open: %d\n", fd);
>  
>  	return fd;
>  }
>  
>  /* Is this an fd for which we are emulating v4l1 ? */
> -static int v4l2_get_index(int fd)
> +int v4l2_get_index(int fd)
>  {
>  	int index;
>  
> 
> commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 12:34:04 2017 +0200
> 
>     Enable ioctl propagation.
> 
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index edc9642..6dab661 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
>  	return 0;
>  }
>  
> +static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
> +{
> +	int i = 0;
> +	int result;
> +	while (1) {
> +		if (devices[index].subdev_fds[i] == -1)
> +			return -1;
> +		printf("g_ctrl failed, trying...\n");
> +		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
> +		printf("subdev %d result %d\n", i, result);
> +		if (result == 0)
> +			return 0;
> +		i++;
> +	}
> +	return -1;
> +}
> +
>  int v4l2_ioctl(int fd, unsigned long int request, ...)
>  {
>  	void *arg;
> @@ -1193,14 +1210,20 @@ no_capture_request:
>  	switch (request) {
>  	case VIDIOC_QUERYCTRL:
>  		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_G_CTRL:
>  		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_S_CTRL:
>  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
> +		if (result == -1)
> +			result = v4l2_propagate_ioctl(index, request, arg);
>  		break;
>  
>  	case VIDIOC_G_EXT_CTRLS:
> 
> 



Thanks,
Mauro

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 4705 bytes --]

Hi!

> > > I don't see why it would be hard to open files or have threads inside
> > > a library. There are several libraries that do that already, specially
> > > the ones designed to be used on multimidia apps.  
> > 
> > Well, This is what the libv4l2 says:
> > 
> >    This file implements libv4l2, which offers v4l2_ prefixed versions
> >    of
> >       open/close/etc. The API is 100% the same as directly opening
> >    /dev/videoX
> >       using regular open/close/etc, the big difference is that format
> >    conversion
> >    
> > but if I open additional files in v4l2_open(), API is no longer the
> > same, as unix open() is defined to open just one file descriptor.
> > 
> > Now. There is autogain support in libv4lconvert, but it expects to use
> > same fd for camera and for the gain... which does not work with
> > subdevs.
> > 
> > Of course, opening subdevs by name like this is not really
> > acceptable. But can you suggest a method that is?
> 
> There are two separate things here:
> 
> 1) Autofoucs for a device that doesn't use subdev API
> 2) libv4l2 support for devices that require MC and subdev API

Actually there are three: 0) autogain. Unfortunately, I need autogain
first before autofocus has a chance...

And that means... bayer10 support for autogain.

Plus, I changed avg_lum to long long. Quick calculation tells me int
could overflow with few megapixel sensor.

Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
longer works.

Regards,
								Pavel

diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
index c6866d6..0b52d0f 100644
--- a/lib/libv4lconvert/processing/autogain.c
+++ b/lib/libv4lconvert/processing/autogain.c
@@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
 	}
 }
 
+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	avg_lum /= 4;
+	return avg_lum;
+}
+
+static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	return avg_lum;
+}
+
 /* auto gain and exposure algorithm based on the knee algorithm described here:
 http://ytse.tricolour.net/docs/LowLightOptimization.html */
 static int autogain_calculate_lookup_tables(
@@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
 	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
+		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
+		break;
+
 	case V4L2_PIX_FMT_SGBRG8:
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
-		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
-			fmt->fmt.pix.width / 4;
-
-		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
-			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
-				avg_lum += *buf++;
-			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
-		}
-		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+		avg_lum = get_luminosity_bayer8(buf, fmt);
 		break;
 
 	case V4L2_PIX_FMT_RGB24:
diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
index b061f50..b98d024 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing.c
+++ b/lib/libv4lconvert/processing/libv4lprocessing.c
@@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		break;



-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

Hi!

> > > I don't see why it would be hard to open files or have threads inside
> > > a library. There are several libraries that do that already, specially
> > > the ones designed to be used on multimidia apps.  
> > 
> > Well, This is what the libv4l2 says:
> > 
> >    This file implements libv4l2, which offers v4l2_ prefixed versions
> >    of
> >       open/close/etc. The API is 100% the same as directly opening
> >    /dev/videoX
> >       using regular open/close/etc, the big difference is that format
> >    conversion
> >    
> > but if I open additional files in v4l2_open(), API is no longer the
> > same, as unix open() is defined to open just one file descriptor.
> > 
> > Now. There is autogain support in libv4lconvert, but it expects to use
> > same fd for camera and for the gain... which does not work with
> > subdevs.
> > 
> > Of course, opening subdevs by name like this is not really
> > acceptable. But can you suggest a method that is?
> 
> There are two separate things here:
> 
> 1) Autofoucs for a device that doesn't use subdev API
> 2) libv4l2 support for devices that require MC and subdev API

Actually there are three: 0) autogain. Unfortunately, I need autogain
first before autofocus has a chance...

And that means... bayer10 support for autogain.

Plus, I changed avg_lum to long long. Quick calculation tells me int
could overflow with few megapixel sensor.

Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
longer works.

Regards,
								Pavel

diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
index c6866d6..0b52d0f 100644
--- a/lib/libv4lconvert/processing/autogain.c
+++ b/lib/libv4lconvert/processing/autogain.c
@@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
 	}
 }
 
+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	avg_lum /= 4;
+	return avg_lum;
+}
+
+static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	return avg_lum;
+}
+
 /* auto gain and exposure algorithm based on the knee algorithm described here:
 http://ytse.tricolour.net/docs/LowLightOptimization.html */
 static int autogain_calculate_lookup_tables(
@@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
 	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
+		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
+		break;
+
 	case V4L2_PIX_FMT_SGBRG8:
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
-		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
-			fmt->fmt.pix.width / 4;
-
-		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
-			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
-				avg_lum += *buf++;
-			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
-		}
-		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+		avg_lum = get_luminosity_bayer8(buf, fmt);
 		break;
 
 	case V4L2_PIX_FMT_RGB24:
diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
index b061f50..b98d024 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing.c
+++ b/lib/libv4lconvert/processing/libv4lprocessing.c
@@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		break;



-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170426/7ba6278a/attachment-0001.sig>

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Ivaylo Dimitrov]

Hi,

On 26.04.2017 16:23, Pavel Machek wrote:
> Hi!
>
>>>> I don't see why it would be hard to open files or have threads inside
>>>> a library. There are several libraries that do that already, specially
>>>> the ones designed to be used on multimidia apps.
>>>
>>> Well, This is what the libv4l2 says:
>>>
>>>    This file implements libv4l2, which offers v4l2_ prefixed versions
>>>    of
>>>       open/close/etc. The API is 100% the same as directly opening
>>>    /dev/videoX
>>>       using regular open/close/etc, the big difference is that format
>>>    conversion
>>>
>>> but if I open additional files in v4l2_open(), API is no longer the
>>> same, as unix open() is defined to open just one file descriptor.
>>>
>>> Now. There is autogain support in libv4lconvert, but it expects to use
>>> same fd for camera and for the gain... which does not work with
>>> subdevs.
>>>
>>> Of course, opening subdevs by name like this is not really
>>> acceptable. But can you suggest a method that is?
>>
>> There are two separate things here:
>>
>> 1) Autofoucs for a device that doesn't use subdev API
>> 2) libv4l2 support for devices that require MC and subdev API
>
> Actually there are three: 0) autogain. Unfortunately, I need autogain
> first before autofocus has a chance...
>
> And that means... bayer10 support for autogain.
>
> Plus, I changed avg_lum to long long. Quick calculation tells me int
> could overflow with few megapixel sensor.
>
> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> longer works.
>
> Regards,
> 								Pavel
>
> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> index c6866d6..0b52d0f 100644
> --- a/lib/libv4lconvert/processing/autogain.c
> +++ b/lib/libv4lconvert/processing/autogain.c
> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>  	}
>  }
>
> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)

That would take some time :). AIUI, we have NEON support in ARM kernels 
(CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert 
the above loop to NEON-optimized when it comes to it? Are there any 
drawbacks in using NEON code in kernel?

> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	avg_lum /= 4;
> +	return avg_lum;
> +}
> +
> +static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)

ditto.

> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	return avg_lum;
> +}
> +
>  /* auto gain and exposure algorithm based on the knee algorithm described here:
>  http://ytse.tricolour.net/docs/LowLightOptimization.html */
>  static int autogain_calculate_lookup_tables(
> @@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
>  	switch (fmt->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
> +		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
> +		break;
> +
>  	case V4L2_PIX_FMT_SGBRG8:
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> -		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> -			fmt->fmt.pix.width / 4;
> -
> -		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> -			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> -				avg_lum += *buf++;
> -			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> -		}
> -		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +		avg_lum = get_luminosity_bayer8(buf, fmt);
>  		break;
>
>  	case V4L2_PIX_FMT_RGB24:
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
> index b061f50..b98d024 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing.c
> +++ b/lib/libv4lconvert/processing/libv4lprocessing.c
> @@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
>  	case V4L2_PIX_FMT_RGB24:
>  	case V4L2_PIX_FMT_BGR24:
>  		break;
>
>
>

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Ivaylo Dimitrov]

Hi,

On 26.04.2017 16:23, Pavel Machek wrote:
> Hi!
>
>>>> I don't see why it would be hard to open files or have threads inside
>>>> a library. There are several libraries that do that already, specially
>>>> the ones designed to be used on multimidia apps.
>>>
>>> Well, This is what the libv4l2 says:
>>>
>>>    This file implements libv4l2, which offers v4l2_ prefixed versions
>>>    of
>>>       open/close/etc. The API is 100% the same as directly opening
>>>    /dev/videoX
>>>       using regular open/close/etc, the big difference is that format
>>>    conversion
>>>
>>> but if I open additional files in v4l2_open(), API is no longer the
>>> same, as unix open() is defined to open just one file descriptor.
>>>
>>> Now. There is autogain support in libv4lconvert, but it expects to use
>>> same fd for camera and for the gain... which does not work with
>>> subdevs.
>>>
>>> Of course, opening subdevs by name like this is not really
>>> acceptable. But can you suggest a method that is?
>>
>> There are two separate things here:
>>
>> 1) Autofoucs for a device that doesn't use subdev API
>> 2) libv4l2 support for devices that require MC and subdev API
>
> Actually there are three: 0) autogain. Unfortunately, I need autogain
> first before autofocus has a chance...
>
> And that means... bayer10 support for autogain.
>
> Plus, I changed avg_lum to long long. Quick calculation tells me int
> could overflow with few megapixel sensor.
>
> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> longer works.
>
> Regards,
> 								Pavel
>
> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> index c6866d6..0b52d0f 100644
> --- a/lib/libv4lconvert/processing/autogain.c
> +++ b/lib/libv4lconvert/processing/autogain.c
> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>  	}
>  }
>
> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)

That would take some time :). AIUI, we have NEON support in ARM kernels 
(CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert 
the above loop to NEON-optimized when it comes to it? Are there any 
drawbacks in using NEON code in kernel?

> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	avg_lum /= 4;
> +	return avg_lum;
> +}
> +
> +static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)

ditto.

> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	return avg_lum;
> +}
> +
>  /* auto gain and exposure algorithm based on the knee algorithm described here:
>  http://ytse.tricolour.net/docs/LowLightOptimization.html */
>  static int autogain_calculate_lookup_tables(
> @@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
>  	switch (fmt->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
> +		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
> +		break;
> +
>  	case V4L2_PIX_FMT_SGBRG8:
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> -		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> -			fmt->fmt.pix.width / 4;
> -
> -		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> -			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> -				avg_lum += *buf++;
> -			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> -		}
> -		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +		avg_lum = get_luminosity_bayer8(buf, fmt);
>  		break;
>
>  	case V4L2_PIX_FMT_RGB24:
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
> index b061f50..b98d024 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing.c
> +++ b/lib/libv4lconvert/processing/libv4lprocessing.c
> @@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
>  	case V4L2_PIX_FMT_RGB24:
>  	case V4L2_PIX_FMT_BGR24:
>  		break;
>
>
>

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2240 bytes --]

Hi!

> >>There are two separate things here:
> >>
> >>1) Autofoucs for a device that doesn't use subdev API
> >>2) libv4l2 support for devices that require MC and subdev API
> >
> >Actually there are three: 0) autogain. Unfortunately, I need autogain
> >first before autofocus has a chance...
> >
> >And that means... bayer10 support for autogain.
> >
> >Plus, I changed avg_lum to long long. Quick calculation tells me int
> >could overflow with few megapixel sensor.
> >
> >Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> >longer works.
> >
> >Regards,
> >								Pavel
> >
> >diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> >index c6866d6..0b52d0f 100644
> >--- a/lib/libv4lconvert/processing/autogain.c
> >+++ b/lib/libv4lconvert/processing/autogain.c
> >@@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
> > 	}
> > }
> >
> >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> >+{
> >+	long long avg_lum = 0;
> >+	int x, y;
> >+	
> >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> >+		fmt->fmt.pix.width / 4;
> >+
> >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> 
> That would take some time :). AIUI, we have NEON support in ARM kernels
> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> using NEON code in kernel?

Well, thanks for offer. This is actualy libv4l2.

But I'd say NEON conversion is not neccessary anytime soon. First,
this is just trying to get average luminosity. We can easily skip
quite a lot of pixels, and still get reasonable answer.

Second, omap3isp actually has a hardware block computing statistics
for us. We just don't use it for simplicity.

(But if you want to play with camera, I'll get you patches; there's
ton of work to be done, both kernel and userspace :-).

Regards,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

Hi!

> >>There are two separate things here:
> >>
> >>1) Autofoucs for a device that doesn't use subdev API
> >>2) libv4l2 support for devices that require MC and subdev API
> >
> >Actually there are three: 0) autogain. Unfortunately, I need autogain
> >first before autofocus has a chance...
> >
> >And that means... bayer10 support for autogain.
> >
> >Plus, I changed avg_lum to long long. Quick calculation tells me int
> >could overflow with few megapixel sensor.
> >
> >Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> >longer works.
> >
> >Regards,
> >								Pavel
> >
> >diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> >index c6866d6..0b52d0f 100644
> >--- a/lib/libv4lconvert/processing/autogain.c
> >+++ b/lib/libv4lconvert/processing/autogain.c
> >@@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
> > 	}
> > }
> >
> >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> >+{
> >+	long long avg_lum = 0;
> >+	int x, y;
> >+	
> >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> >+		fmt->fmt.pix.width / 4;
> >+
> >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> 
> That would take some time :). AIUI, we have NEON support in ARM kernels
> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> using NEON code in kernel?

Well, thanks for offer. This is actualy libv4l2.

But I'd say NEON conversion is not neccessary anytime soon. First,
this is just trying to get average luminosity. We can easily skip
quite a lot of pixels, and still get reasonable answer.

Second, omap3isp actually has a hardware block computing statistics
for us. We just don't use it for simplicity.

(But if you want to play with camera, I'll get you patches; there's
ton of work to be done, both kernel and userspace :-).

Regards,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170427/664aa13a/attachment.sig>

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1.1: Type: text/plain, Size: 2240 bytes --]

Hi!

> >>There are two separate things here:
> >>
> >>1) Autofoucs for a device that doesn't use subdev API
> >>2) libv4l2 support for devices that require MC and subdev API
> >
> >Actually there are three: 0) autogain. Unfortunately, I need autogain
> >first before autofocus has a chance...
> >
> >And that means... bayer10 support for autogain.
> >
> >Plus, I changed avg_lum to long long. Quick calculation tells me int
> >could overflow with few megapixel sensor.
> >
> >Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> >longer works.
> >
> >Regards,
> >								Pavel
> >
> >diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> >index c6866d6..0b52d0f 100644
> >--- a/lib/libv4lconvert/processing/autogain.c
> >+++ b/lib/libv4lconvert/processing/autogain.c
> >@@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
> > 	}
> > }
> >
> >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> >+{
> >+	long long avg_lum = 0;
> >+	int x, y;
> >+	
> >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> >+		fmt->fmt.pix.width / 4;
> >+
> >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> 
> That would take some time :). AIUI, we have NEON support in ARM kernels
> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> using NEON code in kernel?

Well, thanks for offer. This is actualy libv4l2.

But I'd say NEON conversion is not neccessary anytime soon. First,
this is just trying to get average luminosity. We can easily skip
quite a lot of pixels, and still get reasonable answer.

Second, omap3isp actually has a hardware block computing statistics
for us. We just don't use it for simplicity.

(But if you want to play with camera, I'll get you patches; there's
ton of work to be done, both kernel and userspace :-).

Regards,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 176 bytes --]

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Ivaylo Dimitrov]

On 27.04.2017 01:51, Pavel Machek wrote:
> Hi!
>
>>>> There are two separate things here:
>>>>
>>>> 1) Autofoucs for a device that doesn't use subdev API
>>>> 2) libv4l2 support for devices that require MC and subdev API
>>>
>>> Actually there are three: 0) autogain. Unfortunately, I need autogain
>>> first before autofocus has a chance...
>>>
>>> And that means... bayer10 support for autogain.
>>>
>>> Plus, I changed avg_lum to long long. Quick calculation tells me int
>>> could overflow with few megapixel sensor.
>>>
>>> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
>>> longer works.
>>>
>>> Regards,
>>> 								Pavel
>>>
>>> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
>>> index c6866d6..0b52d0f 100644
>>> --- a/lib/libv4lconvert/processing/autogain.c
>>> +++ b/lib/libv4lconvert/processing/autogain.c
>>> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>>> 	}
>>> }
>>>
>>> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
>>> +{
>>> +	long long avg_lum = 0;
>>> +	int x, y;
>>> +	
>>> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
>>> +		fmt->fmt.pix.width / 4;
>>> +
>>> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
>>> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
>>
>> That would take some time :). AIUI, we have NEON support in ARM kernels
>> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
>> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
>> using NEON code in kernel?
>
> Well, thanks for offer. This is actualy libv4l2.
>

Oh, somehow I got confused that this is kernel code :)

> But I'd say NEON conversion is not neccessary anytime soon. First,
> this is just trying to get average luminosity. We can easily skip
> quite a lot of pixels, and still get reasonable answer.
>
> Second, omap3isp actually has a hardware block computing statistics
> for us. We just don't use it for simplicity.
>

Right, I forgot about that.

> (But if you want to play with camera, I'll get you patches; there's
> ton of work to be done, both kernel and userspace :-).

Well, I saw a low hanging fruit I thought I can convert to NEON in a day 
or two, while having some rest from the huge "project" I am devoting all 
my spare time recently (rebasing hildon/maemo 5 on top of devuan 
Jessie). Still, if there is something relatively small to be done, just 
email me and I'll have a look.

Regards,
Ivo

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Ivaylo Dimitrov]

On 27.04.2017 01:51, Pavel Machek wrote:
> Hi!
>
>>>> There are two separate things here:
>>>>
>>>> 1) Autofoucs for a device that doesn't use subdev API
>>>> 2) libv4l2 support for devices that require MC and subdev API
>>>
>>> Actually there are three: 0) autogain. Unfortunately, I need autogain
>>> first before autofocus has a chance...
>>>
>>> And that means... bayer10 support for autogain.
>>>
>>> Plus, I changed avg_lum to long long. Quick calculation tells me int
>>> could overflow with few megapixel sensor.
>>>
>>> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
>>> longer works.
>>>
>>> Regards,
>>> 								Pavel
>>>
>>> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
>>> index c6866d6..0b52d0f 100644
>>> --- a/lib/libv4lconvert/processing/autogain.c
>>> +++ b/lib/libv4lconvert/processing/autogain.c
>>> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>>> 	}
>>> }
>>>
>>> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
>>> +{
>>> +	long long avg_lum = 0;
>>> +	int x, y;
>>> +	
>>> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
>>> +		fmt->fmt.pix.width / 4;
>>> +
>>> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
>>> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
>>
>> That would take some time :). AIUI, we have NEON support in ARM kernels
>> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
>> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
>> using NEON code in kernel?
>
> Well, thanks for offer. This is actualy libv4l2.
>

Oh, somehow I got confused that this is kernel code :)

> But I'd say NEON conversion is not neccessary anytime soon. First,
> this is just trying to get average luminosity. We can easily skip
> quite a lot of pixels, and still get reasonable answer.
>
> Second, omap3isp actually has a hardware block computing statistics
> for us. We just don't use it for simplicity.
>

Right, I forgot about that.

> (But if you want to play with camera, I'll get you patches; there's
> ton of work to be done, both kernel and userspace :-).

Well, I saw a low hanging fruit I thought I can convert to NEON in a day 
or two, while having some rest from the huge "project" I am devoting all 
my spare time recently (rebasing hildon/maemo 5 on top of devuan 
Jessie). Still, if there is something relatively small to be done, just 
email me and I'll have a look.

Regards,
Ivo

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

Hi!

> Oh, somehow I got confused that this is kernel code :)
> 
> >But I'd say NEON conversion is not neccessary anytime soon. First,
> >this is just trying to get average luminosity. We can easily skip
> >quite a lot of pixels, and still get reasonable answer.
> >
> >Second, omap3isp actually has a hardware block computing statistics
> >for us. We just don't use it for simplicity.
> >
> 
> Right, I forgot about that.
> 
> >(But if you want to play with camera, I'll get you patches; there's
> >ton of work to be done, both kernel and userspace :-).
> 
> Well, I saw a low hanging fruit I thought I can convert to NEON in a day or
> two, while having some rest from the huge "project" I am devoting all my
> spare time recently (rebasing hildon/maemo 5 on top of devuan Jessie).
> Still, if there is something relatively small to be done, just email me and
> I'll have a look.

Well, there's a ton of work on camera, and some work on
libcmtspeechdata. The later is rather self-contained.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

Hi!

> Oh, somehow I got confused that this is kernel code :)
> 
> >But I'd say NEON conversion is not neccessary anytime soon. First,
> >this is just trying to get average luminosity. We can easily skip
> >quite a lot of pixels, and still get reasonable answer.
> >
> >Second, omap3isp actually has a hardware block computing statistics
> >for us. We just don't use it for simplicity.
> >
> 
> Right, I forgot about that.
> 
> >(But if you want to play with camera, I'll get you patches; there's
> >ton of work to be done, both kernel and userspace :-).
> 
> Well, I saw a low hanging fruit I thought I can convert to NEON in a day or
> two, while having some rest from the huge "project" I am devoting all my
> spare time recently (rebasing hildon/maemo 5 on top of devuan Jessie).
> Still, if there is something relatively small to be done, just email me and
> I'll have a look.

Well, there's a ton of work on camera, and some work on
libcmtspeechdata. The later is rather self-contained.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170713/01ea4d01/attachment.sig>

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Russell King - ARM Linux]

On Wed, Apr 26, 2017 at 06:43:54PM +0300, Ivaylo Dimitrov wrote:
> >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> >+{
> >+	long long avg_lum = 0;
> >+	int x, y;
> >+	
> >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> >+		fmt->fmt.pix.width / 4;
> >+
> >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> 
> That would take some time :). AIUI, we have NEON support in ARM kernels
> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> using NEON code in kernel?

Using neon without the VFP state saved and restored corrupts userspace's
FP state.  So, you have to save the entire VFP state to use neon in kernel
mode.  There are helper functions for this: kernel_neon_begin() and
kernel_neon_end().

You can't build C code with the compiler believing that neon is available
as the compiler could emit neon instructions in unprotected kernel code.

Note that kernel_neon_begin() is only allowed to be called outside
interrupt context and with preemption disabled.

Given that, do we really want to be walking over multi-megabytes of image
data in the kernel with preemption disabled - it sounds like a recipe for
a very sluggish system.  I think this should (and can only sensibly be
done) in userspace.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Russell King - ARM Linux]

On Wed, Apr 26, 2017 at 06:43:54PM +0300, Ivaylo Dimitrov wrote:
> >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> >+{
> >+	long long avg_lum = 0;
> >+	int x, y;
> >+	
> >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> >+		fmt->fmt.pix.width / 4;
> >+
> >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> 
> That would take some time :). AIUI, we have NEON support in ARM kernels
> (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> using NEON code in kernel?

Using neon without the VFP state saved and restored corrupts userspace's
FP state.  So, you have to save the entire VFP state to use neon in kernel
mode.  There are helper functions for this: kernel_neon_begin() and
kernel_neon_end().

You can't build C code with the compiler believing that neon is available
as the compiler could emit neon instructions in unprotected kernel code.

Note that kernel_neon_begin() is only allowed to be called outside
interrupt context and with preemption disabled.

Given that, do we really want to be walking over multi-megabytes of image
data in the kernel with preemption disabled - it sounds like a recipe for
a very sluggish system.  I think this should (and can only sensibly be
done) in userspace.

-- 
RMK's Patch system: http://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line: currently at 9.6Mbps down 400kbps up
according to speedtest.net.

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1532 bytes --]

On Wed 2017-05-03 20:05:56, Russell King - ARM Linux wrote:
> On Wed, Apr 26, 2017 at 06:43:54PM +0300, Ivaylo Dimitrov wrote:
> > >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> > >+{
> > >+	long long avg_lum = 0;
> > >+	int x, y;
> > >+	
> > >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> > >+		fmt->fmt.pix.width / 4;
> > >+
> > >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> > >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> > 
> > That would take some time :). AIUI, we have NEON support in ARM kernels
> > (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> > above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> > using NEON code in kernel?
> 
> Using neon without the VFP state saved and restored corrupts userspace's
> FP state.  So, you have to save the entire VFP state to use neon in kernel
> mode.  There are helper functions for this: kernel_neon_begin() and
> kernel_neon_end().
...
> Given that, do we really want to be walking over multi-megabytes of image
> data in the kernel with preemption disabled - it sounds like a recipe for
> a very sluggish system.  I think this should (and can only sensibly be
> done) in userspace.

The patch was for libv4l2. (And I explained why we don't need to
overoptimize this.)
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

On Wed 2017-05-03 20:05:56, Russell King - ARM Linux wrote:
> On Wed, Apr 26, 2017 at 06:43:54PM +0300, Ivaylo Dimitrov wrote:
> > >+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> > >+{
> > >+	long long avg_lum = 0;
> > >+	int x, y;
> > >+	
> > >+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> > >+		fmt->fmt.pix.width / 4;
> > >+
> > >+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> > >+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> > 
> > That would take some time :). AIUI, we have NEON support in ARM kernels
> > (CONFIG_KERNEL_MODE_NEON), I wonder if it makes sense (me) to convert the
> > above loop to NEON-optimized when it comes to it? Are there any drawbacks in
> > using NEON code in kernel?
> 
> Using neon without the VFP state saved and restored corrupts userspace's
> FP state.  So, you have to save the entire VFP state to use neon in kernel
> mode.  There are helper functions for this: kernel_neon_begin() and
> kernel_neon_end().
...
> Given that, do we really want to be walking over multi-megabytes of image
> data in the kernel with preemption disabled - it sounds like a recipe for
> a very sluggish system.  I think this should (and can only sensibly be
> done) in userspace.

The patch was for libv4l2. (And I explained why we don't need to
overoptimize this.)
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170503/e1091649/attachment.sig>

[patch, libv4l]: fix integer overflow

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1464 bytes --]

Hi!

This bit me while trying to use absolute exposure time on Nokia N900:

Can someone apply it to libv4l2 tree? Could I get some feedback on the
other patches? Is this the way to submit patches to libv4l2?

Thanks,
								Pavel

commit 0484e39ec05fdc644191e7c334a7ebfff9cb2ec5
Author: Pavel <pavel@ucw.cz>
Date:   Mon May 8 21:52:02 2017 +0200

    Fix integer overflow with EXPOSURE_ABSOLUTE.

diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index e795aee..189fc06 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1776,7 +1776,7 @@ int v4l2_set_control(int fd, int cid, int value)
 		if (qctrl.type == V4L2_CTRL_TYPE_BOOLEAN)
 			ctrl.value = value ? 1 : 0;
 		else
-			ctrl.value = (value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
+			ctrl.value = ((long long) value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
 				qctrl.minimum;
 
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, &ctrl);
@@ -1812,7 +1812,7 @@ int v4l2_get_control(int fd, int cid)
 		if (v4l2_propagate_ioctl(index, VIDIOC_G_CTRL, &ctrl))
 			return -1;
 
-	return ((ctrl.value - qctrl.minimum) * 65535 +
+	return (((long long) ctrl.value - qctrl.minimum) * 65535 +
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: fix integer overflow

[Hans Verkuil]

On 05/09/2017 12:28 AM, Pavel Machek wrote:
> Hi!
> 
> This bit me while trying to use absolute exposure time on Nokia N900:
> 
> Can someone apply it to libv4l2 tree? Could I get some feedback on the
> other patches? Is this the way to submit patches to libv4l2?

Yes, it is. But I do need a Signed-off-by from you.

Regards,

	Hans

> 
> Thanks,
> 								Pavel
> 
> commit 0484e39ec05fdc644191e7c334a7ebfff9cb2ec5
> Author: Pavel <pavel@ucw.cz>
> Date:   Mon May 8 21:52:02 2017 +0200
> 
>     Fix integer overflow with EXPOSURE_ABSOLUTE.
> 
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index e795aee..189fc06 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1776,7 +1776,7 @@ int v4l2_set_control(int fd, int cid, int value)
>  		if (qctrl.type == V4L2_CTRL_TYPE_BOOLEAN)
>  			ctrl.value = value ? 1 : 0;
>  		else
> -			ctrl.value = (value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
> +			ctrl.value = ((long long) value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
>  				qctrl.minimum;
>  
>  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, &ctrl);
> @@ -1812,7 +1812,7 @@ int v4l2_get_control(int fd, int cid)
>  		if (v4l2_propagate_ioctl(index, VIDIOC_G_CTRL, &ctrl))
>  			return -1;
>  
> -	return ((ctrl.value - qctrl.minimum) * 65535 +
> +	return (((long long) ctrl.value - qctrl.minimum) * 65535 +
>  			(qctrl.maximum - qctrl.minimum) / 2) /
>  		(qctrl.maximum - qctrl.minimum);
>  }
> 
> 

Re: [patch, libv4l]: fix integer overflow

[Hans Verkuil]

On 05/09/2017 08:29 AM, Hans Verkuil wrote:
> On 05/09/2017 12:28 AM, Pavel Machek wrote:
>> Hi!
>>
>> This bit me while trying to use absolute exposure time on Nokia N900:
>>
>> Can someone apply it to libv4l2 tree? Could I get some feedback on the
>> other patches? Is this the way to submit patches to libv4l2?
> 
> Yes, it is. But I do need a Signed-off-by from you.

I saw other patches from you for libv4l without a Signed-off-by. Can you
check them and reply with the Signed-off-by line?

Thanks!

	Hans

> 
> Regards,
> 
> 	Hans
> 
>>
>> Thanks,
>> 								Pavel
>>
>> commit 0484e39ec05fdc644191e7c334a7ebfff9cb2ec5
>> Author: Pavel <pavel@ucw.cz>
>> Date:   Mon May 8 21:52:02 2017 +0200
>>
>>     Fix integer overflow with EXPOSURE_ABSOLUTE.
>>
>> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
>> index e795aee..189fc06 100644
>> --- a/lib/libv4l2/libv4l2.c
>> +++ b/lib/libv4l2/libv4l2.c
>> @@ -1776,7 +1776,7 @@ int v4l2_set_control(int fd, int cid, int value)
>>  		if (qctrl.type == V4L2_CTRL_TYPE_BOOLEAN)
>>  			ctrl.value = value ? 1 : 0;
>>  		else
>> -			ctrl.value = (value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
>> +			ctrl.value = ((long long) value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
>>  				qctrl.minimum;
>>  
>>  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, &ctrl);
>> @@ -1812,7 +1812,7 @@ int v4l2_get_control(int fd, int cid)
>>  		if (v4l2_propagate_ioctl(index, VIDIOC_G_CTRL, &ctrl))
>>  			return -1;
>>  
>> -	return ((ctrl.value - qctrl.minimum) * 65535 +
>> +	return (((long long) ctrl.value - qctrl.minimum) * 65535 +
>>  			(qctrl.maximum - qctrl.minimum) / 2) /
>>  		(qctrl.maximum - qctrl.minimum);
>>  }
>>
>>
> 

Re: [patch, libv4l]: fix integer overflow

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 807 bytes --]

On Tue 2017-05-09 08:32:20, Hans Verkuil wrote:
> On 05/09/2017 08:29 AM, Hans Verkuil wrote:
> > On 05/09/2017 12:28 AM, Pavel Machek wrote:
> >> Hi!
> >>
> >> This bit me while trying to use absolute exposure time on Nokia N900:
> >>
> >> Can someone apply it to libv4l2 tree? Could I get some feedback on the
> >> other patches? Is this the way to submit patches to libv4l2?
> > 
> > Yes, it is. But I do need a Signed-off-by from you.
> 
> I saw other patches from you for libv4l without a Signed-off-by. Can you
> check them and reply with the Signed-off-by line?

Thanks for quick reply. Yes, will do.

Best regards,
									Pavel
									
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch, libv4l]: fix typos

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1248 bytes --]


Fix random typos. (I searched, and the ALSA define does not seem to be
used anywhere, so this should not break anything.)
    
Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 0ba0a88..1b06b6f 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -789,7 +789,7 @@ no_capture:
 
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
-	   a fram erate using the S_PARM ioctl after a S_FMT */
+	   a frame rate using the S_PARM ioctl after a S_FMT */
 	if (devices[index].convert)
 		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
 	v4l2_update_fps(index, &parm);
diff --git a/utils/qv4l2/qv4l2.cpp b/utils/qv4l2/qv4l2.cpp
index bf8b85a..d1c89da 100644
--- a/utils/qv4l2/qv4l2.cpp
+++ b/utils/qv4l2/qv4l2.cpp
@@ -17,7 +17,7 @@
  * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
  */
 
-#ifdef ENABLE_ASLA
+#ifdef ENABLE_ALSA
 extern "C" {
 #include "alsa_stream.h"
 }

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch, libv4l]: Add support for GRBG10 format conversion

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2118 bytes --]


Add support for GRBG10 format conversion.
    
Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
index d3d8936..a7376b8 100644
--- a/lib/libv4lconvert/libv4lconvert.c
+++ b/lib/libv4lconvert/libv4lconvert.c
@@ -123,6 +123,7 @@ static const struct v4lconvert_pixfmt supported_src_pixfmts[] = {
 	{ V4L2_PIX_FMT_SGRBG8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_SRGGB8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_STV0680,		 8,	 8,	 8,	1 },
+	{ V4L2_PIX_FMT_SGRBG10,		16,	 8,	 8,	1 },
 	/* compressed bayer */
 	{ V4L2_PIX_FMT_SPCA561,		 0,	 9,	 9,	1 },
 	{ V4L2_PIX_FMT_SN9C10X,		 0,	 9,	 9,	1 },
@@ -694,6 +695,16 @@ unsigned char *v4lconvert_alloc_buffer(int needed,
 	return *buf;
 }
 
+static void v4lconvert_10to8(void *_src, unsigned char *dst, int width, int height)
+{
+	int i;
+	uint16_t *src = _src;
+	
+	for (i=0; i<width*height; i++) {
+		dst[i] = src[i] >> 2;
+	}
+}
+
 int v4lconvert_oom_error(struct v4lconvert_data *data)
 {
 	V4LCONVERT_ERR("could not allocate memory\n");
@@ -867,7 +878,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 #endif
 	case V4L2_PIX_FMT_SN9C2028:
 	case V4L2_PIX_FMT_SQ905C:
-	case V4L2_PIX_FMT_STV0680: { /* Not compressed but needs some shuffling */
+	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10: { /* Not compressed but needs some shuffling */
 		unsigned char *tmpbuf;
 		struct v4l2_format tmpfmt = *fmt;
 
@@ -877,6 +889,11 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			return v4lconvert_oom_error(data);
 
 		switch (src_pix_fmt) {
+		case V4L2_PIX_FMT_SGRBG10:
+			v4lconvert_10to8(src, tmpbuf, width, height);
+			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG8;
+			bytesperline = width;
+			break;
 		case V4L2_PIX_FMT_SPCA561:
 			v4lconvert_decode_spca561(src, tmpbuf, width, height);
 			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGBRG8;

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch, libv4l]: Introduce define for lookup table size

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3552 bytes --]


Make lookup table size configurable at compile-time.
    
Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/lib/libv4lconvert/processing/libv4lprocessing-priv.h b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
index e4a29dd..55e1687 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing-priv.h
+++ b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
@@ -25,6 +25,8 @@
 #include "../libv4lsyscall-priv.h"
 
 #define V4L2PROCESSING_UPDATE_RATE 10
+/* Size of lookup tables */
+#define LSIZE 256
 
 struct v4lprocessing_data {
 	struct v4lcontrol_data *control;
@@ -32,15 +34,15 @@ struct v4lprocessing_data {
 	int do_process;
 	int controls_changed;
 	/* True if any of the lookup tables does not contain
-	   linear 0-255 */
+	   linear 0-LSIZE-1 */
 	int lookup_table_active;
 	/* Counts the number of processed frames until a
 	   V4L2PROCESSING_UPDATE_RATE overflow happens */
 	int lookup_table_update_counter;
 	/* RGB/BGR lookup tables */
-	unsigned char comp1[256];
-	unsigned char green[256];
-	unsigned char comp2[256];
+	unsigned char comp1[LSIZE];
+	unsigned char green[LSIZE];
+	unsigned char comp2[LSIZE];
 	/* Filter private data for filters which need it */
 	/* whitebalance.c data */
 	int green_avg;
@@ -48,7 +50,7 @@ struct v4lprocessing_data {
 	int comp2_avg;
 	/* gamma.c data */
 	int last_gamma;
-	unsigned char gamma_table[256];
+	unsigned char gamma_table[LSIZE];
 	/* autogain.c data */
 	int last_gain_correction;
 };
diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
index b061f50..6d0ad20 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing.c
+++ b/lib/libv4lconvert/processing/libv4lprocessing.c
@@ -74,7 +74,7 @@ static void v4lprocessing_update_lookup_tables(struct v4lprocessing_data *data,
 {
 	int i;
 
-	for (i = 0; i < 256; i++) {
+	for (i = 0; i < LSIZE; i++) {
 		data->comp1[i] = i;
 		data->green[i] = i;
 		data->comp2[i] = i;
diff --git a/lib/libv4lconvert/processing/whitebalance.c b/lib/libv4lconvert/processing/whitebalance.c
index c74069a..2dd33c1 100644
--- a/lib/libv4lconvert/processing/whitebalance.c
+++ b/lib/libv4lconvert/processing/whitebalance.c
@@ -27,7 +27,7 @@
 #include "libv4lprocessing-priv.h"
 #include "../libv4lconvert-priv.h" /* for PIX_FMT defines */
 
-#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
+#define CLIPLSIZE(color) (((color) > LSIZE) ? LSIZE : (((color) < 0) ? 0 : (color)))
 #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
 
 static int whitebalance_active(struct v4lprocessing_data *data)
@@ -111,10 +111,10 @@ static int whitebalance_calculate_lookup_tables_generic(
 
 	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
 
-	for (i = 0; i < 256; i++) {
-		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
-		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
-		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
+	for (i = 0; i < LSIZE; i++) {
+		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
+		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
+		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
 	}
 
 	return 1;

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: Introduce define for lookup table size

[Hans Verkuil]

On 09/05/17 13:04, Pavel Machek wrote:
> 
> Make lookup table size configurable at compile-time.

I don't think I'll take this patch. The problem is that if we really add
support for 10 or 12 bit lookup tables in the future, then just changing
LSIZE isn't enough.

This patch doesn't really add anything as it stands.

Regards,

	Hans

>     
> Signed-off-by: Pavel Machek <pavel@ucw.cz>
> 
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing-priv.h b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
> index e4a29dd..55e1687 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing-priv.h
> +++ b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
> @@ -25,6 +25,8 @@
>  #include "../libv4lsyscall-priv.h"
>  
>  #define V4L2PROCESSING_UPDATE_RATE 10
> +/* Size of lookup tables */
> +#define LSIZE 256
>  
>  struct v4lprocessing_data {
>  	struct v4lcontrol_data *control;
> @@ -32,15 +34,15 @@ struct v4lprocessing_data {
>  	int do_process;
>  	int controls_changed;
>  	/* True if any of the lookup tables does not contain
> -	   linear 0-255 */
> +	   linear 0-LSIZE-1 */
>  	int lookup_table_active;
>  	/* Counts the number of processed frames until a
>  	   V4L2PROCESSING_UPDATE_RATE overflow happens */
>  	int lookup_table_update_counter;
>  	/* RGB/BGR lookup tables */
> -	unsigned char comp1[256];
> -	unsigned char green[256];
> -	unsigned char comp2[256];
> +	unsigned char comp1[LSIZE];
> +	unsigned char green[LSIZE];
> +	unsigned char comp2[LSIZE];
>  	/* Filter private data for filters which need it */
>  	/* whitebalance.c data */
>  	int green_avg;
> @@ -48,7 +50,7 @@ struct v4lprocessing_data {
>  	int comp2_avg;
>  	/* gamma.c data */
>  	int last_gamma;
> -	unsigned char gamma_table[256];
> +	unsigned char gamma_table[LSIZE];
>  	/* autogain.c data */
>  	int last_gain_correction;
>  };
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
> index b061f50..6d0ad20 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing.c
> +++ b/lib/libv4lconvert/processing/libv4lprocessing.c
> @@ -74,7 +74,7 @@ static void v4lprocessing_update_lookup_tables(struct v4lprocessing_data *data,
>  {
>  	int i;
>  
> -	for (i = 0; i < 256; i++) {
> +	for (i = 0; i < LSIZE; i++) {
>  		data->comp1[i] = i;
>  		data->green[i] = i;
>  		data->comp2[i] = i;
> diff --git a/lib/libv4lconvert/processing/whitebalance.c b/lib/libv4lconvert/processing/whitebalance.c
> index c74069a..2dd33c1 100644
> --- a/lib/libv4lconvert/processing/whitebalance.c
> +++ b/lib/libv4lconvert/processing/whitebalance.c
> @@ -27,7 +27,7 @@
>  #include "libv4lprocessing-priv.h"
>  #include "../libv4lconvert-priv.h" /* for PIX_FMT defines */
>  
> -#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
> +#define CLIPLSIZE(color) (((color) > LSIZE) ? LSIZE : (((color) < 0) ? 0 : (color)))
>  #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
>  
>  static int whitebalance_active(struct v4lprocessing_data *data)
> @@ -111,10 +111,10 @@ static int whitebalance_calculate_lookup_tables_generic(
>  
>  	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
>  
> -	for (i = 0; i < 256; i++) {
> -		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
> -		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
> -		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
> +	for (i = 0; i < LSIZE; i++) {
> +		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
> +		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
> +		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
>  	}
>  
>  	return 1;
> 

Re: [patch, libv4l]: Introduce define for lookup table size

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2105 bytes --]

Hi!

> > Make lookup table size configurable at compile-time.
> 
> I don't think I'll take this patch. The problem is that if we really add
> support for 10 or 12 bit lookup tables in the future, then just changing
> LSIZE isn't enough.
> 
> This patch doesn't really add anything as it stands.

Well, currently we have 256, 255 and 0xff sprinkled through the code,
when it means to say "lookup table size". That is quite wrong (because
you can't really grep "what depends on the table size).

And BTW with the LSIZE set to 1024, 10 bit processing seems to
work. So it is already useful, at least for me.

But now I noticed the patch is subtly wrong:

> > -#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
> > +#define CLIPLSIZE(color) (((color) > LSIZE) ? LSIZE : (((color) <
0) ? 0 : (color)))

This should be LSIZE-1.

So I need to adjust the patch. But I'd still like you to take (fixed
version) for documentation purposes...

Best regards,
									Pavel

> >  #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
> >  
> >  static int whitebalance_active(struct v4lprocessing_data *data)
> > @@ -111,10 +111,10 @@ static int whitebalance_calculate_lookup_tables_generic(
> >  
> >  	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
> >  
> > -	for (i = 0; i < 256; i++) {
> > -		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
> > -		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
> > -		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
> > +	for (i = 0; i < LSIZE; i++) {
> > +		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
> > +		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
> > +		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
> >  	}
> >  
> >  	return 1;
> > 

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: Introduce define for lookup table size

[Hans Verkuil]

On 16/05/17 14:45, Pavel Machek wrote:
> Hi!
> 
>>> Make lookup table size configurable at compile-time.
>>
>> I don't think I'll take this patch. The problem is that if we really add
>> support for 10 or 12 bit lookup tables in the future, then just changing
>> LSIZE isn't enough.
>>
>> This patch doesn't really add anything as it stands.
> 
> Well, currently we have 256, 255 and 0xff sprinkled through the code,
> when it means to say "lookup table size". That is quite wrong (because
> you can't really grep "what depends on the table size).
> 
> And BTW with the LSIZE set to 1024, 10 bit processing seems to
> work. So it is already useful, at least for me.
> 
> But now I noticed the patch is subtly wrong:
> 
>>> -#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
>>> +#define CLIPLSIZE(color) (((color) > LSIZE) ? LSIZE : (((color) <
> 0) ? 0 : (color)))
> 
> This should be LSIZE-1.
> 
> So I need to adjust the patch. But I'd still like you to take (fixed
> version) for documentation purposes...

I much rather do this as part of a longer series that actually adds 10 bit support.

The problem is that adding support for 10 bit doesn't mean you can just use LSIZE
all the time since you still need support for 8 bit as well.

E.g. CLIPLSIZE makes no sense, I would expect to see a CLIP256 and a CLIP1024.

So it becomes a bit more complex than just adding an LSIZE define.

Regards,

	Hans

> 
> Best regards,
> 									Pavel
> 
>>>  #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
>>>  
>>>  static int whitebalance_active(struct v4lprocessing_data *data)
>>> @@ -111,10 +111,10 @@ static int whitebalance_calculate_lookup_tables_generic(
>>>  
>>>  	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
>>>  
>>> -	for (i = 0; i < 256; i++) {
>>> -		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
>>> -		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
>>> -		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
>>> +	for (i = 0; i < LSIZE; i++) {
>>> +		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
>>> +		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
>>> +		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
>>>  	}
>>>  
>>>  	return 1;
>>>
> 

Re: [patch, libv4l]: Introduce define for lookup table size

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2926 bytes --]

Hi!

> >>> Make lookup table size configurable at compile-time.
> >>
> >> I don't think I'll take this patch. The problem is that if we really add
> >> support for 10 or 12 bit lookup tables in the future, then just changing
> >> LSIZE isn't enough.
> >>
> >> This patch doesn't really add anything as it stands.
> > 
> > Well, currently we have 256, 255 and 0xff sprinkled through the code,
> > when it means to say "lookup table size". That is quite wrong (because
> > you can't really grep "what depends on the table size).
> > 
> > And BTW with the LSIZE set to 1024, 10 bit processing seems to
> > work. So it is already useful, at least for me.
> > 
> > But now I noticed the patch is subtly wrong:
> > 
> >>> -#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
> >>> +#define CLIPLSIZE(color) (((color) > LSIZE) ? LSIZE : (((color) <
> > 0) ? 0 : (color)))
> > 
> > This should be LSIZE-1.
> > 
> > So I need to adjust the patch. But I'd still like you to take (fixed
> > version) for documentation purposes...
> 
> I much rather do this as part of a longer series that actually adds 10 bit support.
> 
> The problem is that adding support for 10 bit doesn't mean you can just use LSIZE
> all the time since you still need support for 8 bit as well.
> 
> E.g. CLIPLSIZE makes no sense, I would expect to see a CLIP256 and a CLIP1024.
> 
> So it becomes a bit more complex than just adding an LSIZE define.

Yes, proper 10 bit support will be more complex (and I'm not sure if
I'll be able to do it, I might need help there). OTOH... table size is
used at 7 places in three different files, and 256 is _very_ common
constant.

So IMO this makes sense regardless of full 10-bit support.

Best regards,
									Pavel


> >>>  #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
> >>>  
> >>>  static int whitebalance_active(struct v4lprocessing_data *data)
> >>> @@ -111,10 +111,10 @@ static int whitebalance_calculate_lookup_tables_generic(
> >>>  
> >>>  	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
> >>>  
> >>> -	for (i = 0; i < 256; i++) {
> >>> -		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
> >>> -		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
> >>> -		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
> >>> +	for (i = 0; i < LSIZE; i++) {
> >>> +		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
> >>> +		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
> >>> +		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
> >>>  	}
> >>>  
> >>>  	return 1;
> >>>
> > 

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[libv4l]: How to do 10-bit support? Stand-alone conversions?

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2097 bytes --]

Hi!

> I much rather do this as part of a longer series that actually adds 10 bit support.

> The problem is that adding support for 10 bit doesn't mean you can just use LSIZE
> all the time since you still need support for 8 bit as well.

Heh. I was afraid to hear that. I can probably support 10-bits for
white balance somehow.

How would complete support for 10-bits work? Introduce RGB48 and
modify processing to work in 16-bits?

And the big question: is it possible to do processing with
libv4lconvert, without actually have v4l device available?

For example... I do have a simple camera application. In a viewfinder
mode, it has to work at GRBG10 format, because any conversion is just
too slow. We'd also want to save raw GRBG10 image to disk for raw
processing. So far so good. But we'd also want to save jpeg, which
means converting existing buffer to RGB24, applying white balance (and
maybe bad-pixel, lens shading, vignetting compensation) and saving
jpeg.

I'm trying to use this for conversion:

static void convert_rgb(struct dev_info *dev, struct v4l2_format sfmt, void *buf, struct v4l2_f\
ormat dfmt, void *buf2, int wb)
{
        struct v4lconvert_data *data = v4lconvert_create(dev->fd);
        int res;

        printf("Converting first.");
        if (wb) {
                struct v4l2_control ctrl;
                ctrl.id = V4L2_CID_AUTO_WHITE_BALANCE;
                ctrl.value = 1;
		v4lconvert_vidioc_s_ctrl(data, &ctrl);
	}
        res = v4lconvert_convert(data, &sfmt, &dfmt, buf, SIZE, buf2, SIZE);
	printf("Converting: %d\n", res);
        v4lconvert_destroy(data);
}

but

1) it feels like improper use of internal functions.

2) it crashes when I attempt to do white balance processing.

Is there an interface I could use? Should I create interface for
v4lconvert_ for this kind of processing? Any preferences how it should
look like?

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 8101 bytes --]

Hi!

What about something like this?

Pass map of controls and expects fds to libv4l2...
 
+static struct v4l2_controls_map map = {
+  .num_fds = 2,
+  .num_controls = 3,
+  .map = { [0] = { .control = 0x00980913, .fd = 1 },
+	   [1] = { .control = V4L2_CID_EXPOSURE_ABSOLUTE, .fd = 1 },
+	   [2] = { .control = V4L2_CID_FOCUS_ABSOLUTE, .fd = 2 },
+	   
+         },
+};
+
+static void open_chk(char *name, int fd)
+{
+	int f = open(name, O_RDWR);
+
+	if (fd != f) {
+	  printf("Unexpected error %m opening %s\n", name);
+	  exit(1);
+	}
+}
+
+static void cam_open_n900(struct dev_info *dev)
+{
+	struct v4l2_format *fmt = &dev->fmt;
+	int fd;
+	
+	map.main_fd = open("/dev/video_ccdc", O_RDWR);
+
+	open_chk("/dev/video_sensor", map.main_fd+1);
+	open_chk("/dev/video_focus", map.main_fd+2);
+	//	open_chk("/dev/video_flash", map.main_fd+3);
+
+	v4l2_open_pipeline(&map, 0);
+	dev->fd = map.main_fd;
+

...so you can use it on complex devices. Tested on my N900.

I guess later helper would be added that would parse some kind of
descritption file and do open_pipeline(). But.. lets solve that
next. In the first place, it would be nice to have libv4l2 usable on
complex devices.

Best regards,
							Pavel
Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
index ea1870d..6220dfd 100644
--- a/lib/include/libv4l2.h
+++ b/lib/include/libv4l2.h
@@ -109,6 +109,23 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
    (note the fd is left open in this case). */
 LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
 
+struct v4l2_control_map {
+	unsigned long control;
+	int fd;
+};
+
+struct v4l2_controls_map {
+	int main_fd;
+	int num_fds;
+	int num_controls;
+	struct v4l2_control_map map[];
+};
+
+LIBV4L_PUBLIC int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
+
+LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
+
+
 #ifdef __cplusplus
 }
 #endif /* __cplusplus */
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 1924c91..ebe5dad 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -104,6 +104,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+	struct v4l2_controls_map *map;
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 2db25d1..b3ae70b 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -787,6 +787,8 @@ no_capture:
 	if (index >= devices_used)
 		devices_used = index + 1;
 
+	devices[index].map = NULL;
+
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
 	   a frame rate using the S_PARM ioctl after a S_FMT */
@@ -1056,12 +1058,39 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+int v4l2_get_fd_for_control(int fd, unsigned long control)
+{
+	int index = v4l2_get_index(fd);
+	struct v4l2_controls_map *map = devices[index].map;
+	int lo = 0;
+	int hi = map->num_controls;
+
+	while (lo < hi) {
+		int i = (lo + hi) / 2;
+		if (map->map[i].control == control) {
+			return map->map[i].fd + fd;
+		}
+		if (map->map[i].control > control) {
+			hi = i;
+			continue;
+		}
+		if (map->map[i].control < control) {
+			lo = i+1;
+			continue;
+		}
+		printf("Bad: impossible condition in binary search\n");
+		exit(1);
+	}
+	return fd;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
 	va_list ap;
 	int result, index, saved_err;
-	int is_capture_request = 0, stream_needs_locking = 0;
+	int is_capture_request = 0, stream_needs_locking = 0, 
+	    is_subdev_request = 0;
 
 	va_start(ap, request);
 	arg = va_arg(ap, void *);
@@ -1076,18 +1105,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	   ioctl, causing it to get sign extended, depending upon this behavior */
 	request = (unsigned int)request;
 
+	/* FIXME */
 	if (devices[index].convert == NULL)
 		goto no_capture_request;
 
 	/* Is this a capture request and do we need to take the stream lock? */
 	switch (request) {
-	case VIDIOC_QUERYCAP:
 	case VIDIOC_QUERYCTRL:
 	case VIDIOC_G_CTRL:
 	case VIDIOC_S_CTRL:
 	case VIDIOC_G_EXT_CTRLS:
-	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_S_EXT_CTRLS:
+		is_subdev_request = 1;
+	case VIDIOC_QUERYCAP:
+	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_ENUM_FRAMESIZES:
 	case VIDIOC_ENUM_FRAMEINTERVALS:
 		is_capture_request = 1;
@@ -1151,10 +1182,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	}
 
 	if (!is_capture_request) {
+	  int sub_fd;
 no_capture_request:
+		  sub_fd = fd;
+		if (is_subdev_request) {
+		  sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
+		}
 		result = devices[index].dev_ops->ioctl(
 				devices[index].dev_ops_priv,
-				fd, request, arg);
+				sub_fd, request, arg);
 		saved_err = errno;
 		v4l2_log_ioctl(request, arg, result);
 		errno = saved_err;
@@ -1782,3 +1818,28 @@ int v4l2_get_control(int fd, int cid)
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }
+
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
+{
+	int index;
+	int i;
+
+	for (i=0; i<map->num_controls; i++) {
+	  printf("%lx %d\n", map->map[i].control, map->map[i].fd);
+	  if (map->map[i].fd <= 0) {
+	    printf("Bad fd in map\n");
+	    return -1;
+	  }
+	  if (i>=1 && map->map[i].control <= map->map[i-1].control) {
+	    printf("Not sorted\n");
+	    return -1;
+	  }
+	}
+
+	v4l2_fd_open(map->main_fd, v4l2_flags);
+	index = v4l2_get_index(map->main_fd);
+	devices[index].map = map;
+	return 0;
+}
+
diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
index 1e784ed..1963e7e 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.c
+++ b/lib/libv4lconvert/control/libv4lcontrol.c
@@ -865,6 +865,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 	struct v4l2_queryctrl *ctrl = arg;
 	int retval;
 	uint32_t orig_id = ctrl->id;
+	int fd;
 
 	/* if we have an exact match return it */
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
@@ -874,8 +875,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
 	/* find out what the kernel driver would respond. */
-	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_QUERYCTRL, arg);
 
 	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
@@ -905,6 +907,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -913,7 +916,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_G_CTRL, arg);
 }
 
@@ -996,6 +1000,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -1010,7 +1015,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_S_CTRL, arg);
 }
 


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Hans Verkuil]

Hi Pavel,

I really don't want to add functions for this to libv4l2. That's just a
quick hack. The real solution is to parse this from a config file. But
that is a lot more work and it is something that needs to be designed
properly.

And that requires someone to put in the time and effort...

Regards,

	Hans

On 03/16/2018 09:55 PM, Pavel Machek wrote:
> Hi!
> 
> What about something like this?
> 
> Pass map of controls and expects fds to libv4l2...
>  
> +static struct v4l2_controls_map map = {
> +  .num_fds = 2,
> +  .num_controls = 3,
> +  .map = { [0] = { .control = 0x00980913, .fd = 1 },
> +	   [1] = { .control = V4L2_CID_EXPOSURE_ABSOLUTE, .fd = 1 },
> +	   [2] = { .control = V4L2_CID_FOCUS_ABSOLUTE, .fd = 2 },
> +	   
> +         },
> +};
> +
> +static void open_chk(char *name, int fd)
> +{
> +	int f = open(name, O_RDWR);
> +
> +	if (fd != f) {
> +	  printf("Unexpected error %m opening %s\n", name);
> +	  exit(1);
> +	}
> +}
> +
> +static void cam_open_n900(struct dev_info *dev)
> +{
> +	struct v4l2_format *fmt = &dev->fmt;
> +	int fd;
> +	
> +	map.main_fd = open("/dev/video_ccdc", O_RDWR);
> +
> +	open_chk("/dev/video_sensor", map.main_fd+1);
> +	open_chk("/dev/video_focus", map.main_fd+2);
> +	//	open_chk("/dev/video_flash", map.main_fd+3);
> +
> +	v4l2_open_pipeline(&map, 0);
> +	dev->fd = map.main_fd;
> +
> 
> ...so you can use it on complex devices. Tested on my N900.
> 
> I guess later helper would be added that would parse some kind of
> descritption file and do open_pipeline(). But.. lets solve that
> next. In the first place, it would be nice to have libv4l2 usable on
> complex devices.
> 
> Best regards,
> 							Pavel
> Signed-off-by: Pavel Machek <pavel@ucw.cz>
> 
> diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
> index ea1870d..6220dfd 100644
> --- a/lib/include/libv4l2.h
> +++ b/lib/include/libv4l2.h
> @@ -109,6 +109,23 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
>     (note the fd is left open in this case). */
>  LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
>  
> +struct v4l2_control_map {
> +	unsigned long control;
> +	int fd;
> +};
> +
> +struct v4l2_controls_map {
> +	int main_fd;
> +	int num_fds;
> +	int num_controls;
> +	struct v4l2_control_map map[];
> +};
> +
> +LIBV4L_PUBLIC int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
> +
> +LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
> +
> +
>  #ifdef __cplusplus
>  }
>  #endif /* __cplusplus */
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 1924c91..ebe5dad 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -104,6 +104,7 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +	struct v4l2_controls_map *map;
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 2db25d1..b3ae70b 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -787,6 +787,8 @@ no_capture:
>  	if (index >= devices_used)
>  		devices_used = index + 1;
>  
> +	devices[index].map = NULL;
> +
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
>  	   a frame rate using the S_PARM ioctl after a S_FMT */
> @@ -1056,12 +1058,39 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
>  	return 0;
>  }
>  
> +int v4l2_get_fd_for_control(int fd, unsigned long control)
> +{
> +	int index = v4l2_get_index(fd);
> +	struct v4l2_controls_map *map = devices[index].map;
> +	int lo = 0;
> +	int hi = map->num_controls;
> +
> +	while (lo < hi) {
> +		int i = (lo + hi) / 2;
> +		if (map->map[i].control == control) {
> +			return map->map[i].fd + fd;
> +		}
> +		if (map->map[i].control > control) {
> +			hi = i;
> +			continue;
> +		}
> +		if (map->map[i].control < control) {
> +			lo = i+1;
> +			continue;
> +		}
> +		printf("Bad: impossible condition in binary search\n");
> +		exit(1);
> +	}
> +	return fd;
> +}
> +
>  int v4l2_ioctl(int fd, unsigned long int request, ...)
>  {
>  	void *arg;
>  	va_list ap;
>  	int result, index, saved_err;
> -	int is_capture_request = 0, stream_needs_locking = 0;
> +	int is_capture_request = 0, stream_needs_locking = 0, 
> +	    is_subdev_request = 0;
>  
>  	va_start(ap, request);
>  	arg = va_arg(ap, void *);
> @@ -1076,18 +1105,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
>  	   ioctl, causing it to get sign extended, depending upon this behavior */
>  	request = (unsigned int)request;
>  
> +	/* FIXME */
>  	if (devices[index].convert == NULL)
>  		goto no_capture_request;
>  
>  	/* Is this a capture request and do we need to take the stream lock? */
>  	switch (request) {
> -	case VIDIOC_QUERYCAP:
>  	case VIDIOC_QUERYCTRL:
>  	case VIDIOC_G_CTRL:
>  	case VIDIOC_S_CTRL:
>  	case VIDIOC_G_EXT_CTRLS:
> -	case VIDIOC_TRY_EXT_CTRLS:
>  	case VIDIOC_S_EXT_CTRLS:
> +		is_subdev_request = 1;
> +	case VIDIOC_QUERYCAP:
> +	case VIDIOC_TRY_EXT_CTRLS:
>  	case VIDIOC_ENUM_FRAMESIZES:
>  	case VIDIOC_ENUM_FRAMEINTERVALS:
>  		is_capture_request = 1;
> @@ -1151,10 +1182,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
>  	}
>  
>  	if (!is_capture_request) {
> +	  int sub_fd;
>  no_capture_request:
> +		  sub_fd = fd;
> +		if (is_subdev_request) {
> +		  sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
> +		}
>  		result = devices[index].dev_ops->ioctl(
>  				devices[index].dev_ops_priv,
> -				fd, request, arg);
> +				sub_fd, request, arg);
>  		saved_err = errno;
>  		v4l2_log_ioctl(request, arg, result);
>  		errno = saved_err;
> @@ -1782,3 +1818,28 @@ int v4l2_get_control(int fd, int cid)
>  			(qctrl.maximum - qctrl.minimum) / 2) /
>  		(qctrl.maximum - qctrl.minimum);
>  }
> +
> +
> +int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
> +{
> +	int index;
> +	int i;
> +
> +	for (i=0; i<map->num_controls; i++) {
> +	  printf("%lx %d\n", map->map[i].control, map->map[i].fd);
> +	  if (map->map[i].fd <= 0) {
> +	    printf("Bad fd in map\n");
> +	    return -1;
> +	  }
> +	  if (i>=1 && map->map[i].control <= map->map[i-1].control) {
> +	    printf("Not sorted\n");
> +	    return -1;
> +	  }
> +	}
> +
> +	v4l2_fd_open(map->main_fd, v4l2_flags);
> +	index = v4l2_get_index(map->main_fd);
> +	devices[index].map = map;
> +	return 0;
> +}
> +
> diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
> index 1e784ed..1963e7e 100644
> --- a/lib/libv4lconvert/control/libv4lcontrol.c
> +++ b/lib/libv4lconvert/control/libv4lcontrol.c
> @@ -865,6 +865,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
>  	struct v4l2_queryctrl *ctrl = arg;
>  	int retval;
>  	uint32_t orig_id = ctrl->id;
> +	int fd;
>  
>  	/* if we have an exact match return it */
>  	for (i = 0; i < V4LCONTROL_COUNT; i++)
> @@ -874,8 +875,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
>  			return 0;
>  		}
>  
> +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
>  	/* find out what the kernel driver would respond. */
> -	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> +	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
>  			VIDIOC_QUERYCTRL, arg);
>  
>  	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
> @@ -905,6 +907,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
>  {
>  	int i;
>  	struct v4l2_control *ctrl = arg;
> +	int fd;
>  
>  	for (i = 0; i < V4LCONTROL_COUNT; i++)
>  		if ((data->controls & (1 << i)) &&
> @@ -913,7 +916,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
>  			return 0;
>  		}
>  
> -	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
> +	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
>  			VIDIOC_G_CTRL, arg);
>  }
>  
> @@ -996,6 +1000,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
>  {
>  	int i;
>  	struct v4l2_control *ctrl = arg;
> +	int fd;
>  
>  	for (i = 0; i < V4LCONTROL_COUNT; i++)
>  		if ((data->controls & (1 << i)) &&
> @@ -1010,7 +1015,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
>  			return 0;
>  		}
>  
> -	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
> +	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
>  			VIDIOC_S_CTRL, arg);
>  }
>  
> 
> 

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 9070 bytes --]

Hi!

> I really don't want to add functions for this to libv4l2. That's just a
> quick hack. The real solution is to parse this from a config
> file. But

No, this is not a quick hack. These are functions that will eventually
be used by the config parser. (Oh and they allow me to use camera on
complex hardware, but...).

Hmm, I have mentioned that already. See quoted text below. 

> that is a lot more work and it is something that needs to be designed
> properly.
> 
> And that requires someone to put in the time and effort...

Which is what I'm trying to do. But some cooperation from your side is
needed, too. I acknowledged some kind of parser is needed. I can
do that. Are you willing to cooperate?

But I need your feedback on the parts below. We can bikeshed about the
parser later.

Do they look acceptable? Did I hook up right functions in acceptable
way?

If so, yes, I can proceed with parser.

Best regards,
							Pavel


> > ...so you can use it on complex devices. Tested on my N900.
> > 
> > I guess later helper would be added that would parse some kind of
> > descritption file and do open_pipeline(). But.. lets solve that
> > next. In the first place, it would be nice to have libv4l2 usable on
> > complex devices.
> > 
> > Best regards,
> > 							Pavel
> > Signed-off-by: Pavel Machek <pavel@ucw.cz>
> > 
> > diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
> > index ea1870d..6220dfd 100644
> > --- a/lib/include/libv4l2.h
> > +++ b/lib/include/libv4l2.h
> > @@ -109,6 +109,23 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
> >     (note the fd is left open in this case). */
> >  LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
> >  
> > +struct v4l2_control_map {
> > +	unsigned long control;
> > +	int fd;
> > +};
> > +
> > +struct v4l2_controls_map {
> > +	int main_fd;
> > +	int num_fds;
> > +	int num_controls;
> > +	struct v4l2_control_map map[];
> > +};
> > +
> > +LIBV4L_PUBLIC int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
> > +
> > +LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
> > +
> > +
> >  #ifdef __cplusplus
> >  }
> >  #endif /* __cplusplus */
> > diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> > index 1924c91..ebe5dad 100644
> > --- a/lib/libv4l2/libv4l2-priv.h
> > +++ b/lib/libv4l2/libv4l2-priv.h
> > @@ -104,6 +104,7 @@ struct v4l2_dev_info {
> >  	void *plugin_library;
> >  	void *dev_ops_priv;
> >  	const struct libv4l_dev_ops *dev_ops;
> > +	struct v4l2_controls_map *map;
> >  };
> >  
> >  /* From v4l2-plugin.c */
> > diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> > index 2db25d1..b3ae70b 100644
> > --- a/lib/libv4l2/libv4l2.c
> > +++ b/lib/libv4l2/libv4l2.c
> > @@ -787,6 +787,8 @@ no_capture:
> >  	if (index >= devices_used)
> >  		devices_used = index + 1;
> >  
> > +	devices[index].map = NULL;
> > +
> >  	/* Note we always tell v4lconvert to optimize src fmt selection for
> >  	   our default fps, the only exception is the app explicitly selecting
> >  	   a frame rate using the S_PARM ioctl after a S_FMT */
> > @@ -1056,12 +1058,39 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
> >  	return 0;
> >  }
> >  
> > +int v4l2_get_fd_for_control(int fd, unsigned long control)
> > +{
> > +	int index = v4l2_get_index(fd);
> > +	struct v4l2_controls_map *map = devices[index].map;
> > +	int lo = 0;
> > +	int hi = map->num_controls;
> > +
> > +	while (lo < hi) {
> > +		int i = (lo + hi) / 2;
> > +		if (map->map[i].control == control) {
> > +			return map->map[i].fd + fd;
> > +		}
> > +		if (map->map[i].control > control) {
> > +			hi = i;
> > +			continue;
> > +		}
> > +		if (map->map[i].control < control) {
> > +			lo = i+1;
> > +			continue;
> > +		}
> > +		printf("Bad: impossible condition in binary search\n");
> > +		exit(1);
> > +	}
> > +	return fd;
> > +}
> > +
> >  int v4l2_ioctl(int fd, unsigned long int request, ...)
> >  {
> >  	void *arg;
> >  	va_list ap;
> >  	int result, index, saved_err;
> > -	int is_capture_request = 0, stream_needs_locking = 0;
> > +	int is_capture_request = 0, stream_needs_locking = 0, 
> > +	    is_subdev_request = 0;
> >  
> >  	va_start(ap, request);
> >  	arg = va_arg(ap, void *);
> > @@ -1076,18 +1105,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
> >  	   ioctl, causing it to get sign extended, depending upon this behavior */
> >  	request = (unsigned int)request;
> >  
> > +	/* FIXME */
> >  	if (devices[index].convert == NULL)
> >  		goto no_capture_request;
> >  
> >  	/* Is this a capture request and do we need to take the stream lock? */
> >  	switch (request) {
> > -	case VIDIOC_QUERYCAP:
> >  	case VIDIOC_QUERYCTRL:
> >  	case VIDIOC_G_CTRL:
> >  	case VIDIOC_S_CTRL:
> >  	case VIDIOC_G_EXT_CTRLS:
> > -	case VIDIOC_TRY_EXT_CTRLS:
> >  	case VIDIOC_S_EXT_CTRLS:
> > +		is_subdev_request = 1;
> > +	case VIDIOC_QUERYCAP:
> > +	case VIDIOC_TRY_EXT_CTRLS:
> >  	case VIDIOC_ENUM_FRAMESIZES:
> >  	case VIDIOC_ENUM_FRAMEINTERVALS:
> >  		is_capture_request = 1;
> > @@ -1151,10 +1182,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
> >  	}
> >  
> >  	if (!is_capture_request) {
> > +	  int sub_fd;
> >  no_capture_request:
> > +		  sub_fd = fd;
> > +		if (is_subdev_request) {
> > +		  sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
> > +		}
> >  		result = devices[index].dev_ops->ioctl(
> >  				devices[index].dev_ops_priv,
> > -				fd, request, arg);
> > +				sub_fd, request, arg);
> >  		saved_err = errno;
> >  		v4l2_log_ioctl(request, arg, result);
> >  		errno = saved_err;
> > @@ -1782,3 +1818,28 @@ int v4l2_get_control(int fd, int cid)
> >  			(qctrl.maximum - qctrl.minimum) / 2) /
> >  		(qctrl.maximum - qctrl.minimum);
> >  }
> > +
> > +
> > +int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
> > +{
> > +	int index;
> > +	int i;
> > +
> > +	for (i=0; i<map->num_controls; i++) {
> > +	  printf("%lx %d\n", map->map[i].control, map->map[i].fd);
> > +	  if (map->map[i].fd <= 0) {
> > +	    printf("Bad fd in map\n");
> > +	    return -1;
> > +	  }
> > +	  if (i>=1 && map->map[i].control <= map->map[i-1].control) {
> > +	    printf("Not sorted\n");
> > +	    return -1;
> > +	  }
> > +	}
> > +
> > +	v4l2_fd_open(map->main_fd, v4l2_flags);
> > +	index = v4l2_get_index(map->main_fd);
> > +	devices[index].map = map;
> > +	return 0;
> > +}
> > +
> > diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
> > index 1e784ed..1963e7e 100644
> > --- a/lib/libv4lconvert/control/libv4lcontrol.c
> > +++ b/lib/libv4lconvert/control/libv4lcontrol.c
> > @@ -865,6 +865,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
> >  	struct v4l2_queryctrl *ctrl = arg;
> >  	int retval;
> >  	uint32_t orig_id = ctrl->id;
> > +	int fd;
> >  
> >  	/* if we have an exact match return it */
> >  	for (i = 0; i < V4LCONTROL_COUNT; i++)
> > @@ -874,8 +875,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
> >  			return 0;
> >  		}
> >  
> > +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
> >  	/* find out what the kernel driver would respond. */
> > -	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> > +	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
> >  			VIDIOC_QUERYCTRL, arg);
> >  
> >  	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
> > @@ -905,6 +907,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
> >  {
> >  	int i;
> >  	struct v4l2_control *ctrl = arg;
> > +	int fd;
> >  
> >  	for (i = 0; i < V4LCONTROL_COUNT; i++)
> >  		if ((data->controls & (1 << i)) &&
> > @@ -913,7 +916,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
> >  			return 0;
> >  		}
> >  
> > -	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> > +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
> > +	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
> >  			VIDIOC_G_CTRL, arg);
> >  }
> >  
> > @@ -996,6 +1000,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
> >  {
> >  	int i;
> >  	struct v4l2_control *ctrl = arg;
> > +	int fd;
> >  
> >  	for (i = 0; i < V4LCONTROL_COUNT; i++)
> >  		if ((data->controls & (1 << i)) &&
> > @@ -1010,7 +1015,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
> >  			return 0;
> >  		}
> >  
> > -	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
> > +	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
> > +	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
> >  			VIDIOC_S_CTRL, arg);
> >  }
> >  
> > 
> > 

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Mon, 19 Mar 2018 11:23:54 +0100
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > I really don't want to add functions for this to libv4l2. That's just a
> > quick hack. The real solution is to parse this from a config
> > file. But  
> 
> No, this is not a quick hack. These are functions that will eventually
> be used by the config parser. (Oh and they allow me to use camera on
> complex hardware, but...).
> 
> Hmm, I have mentioned that already. See quoted text below. 
> 
> > that is a lot more work and it is something that needs to be designed
> > properly.
> > 
> > And that requires someone to put in the time and effort...  
> 
> Which is what I'm trying to do. But some cooperation from your side is
> needed, too. I acknowledged some kind of parser is needed. I can
> do that. Are you willing to cooperate?
> 
> But I need your feedback on the parts below. We can bikeshed about the
> parser later.
> 
> Do they look acceptable? Did I hook up right functions in acceptable
> way?
> 
> If so, yes, I can proceed with parser.
> 
> Best regards,
> 							Pavel


Pavel,

I appreciate your efforts of adding support for mc-based devices to
libv4l.

I guess the main poin that Hans is pointing is that we should take
extra care in order to avoid adding new symbols to libv4l ABI/API
without being sure that they'll be needed in long term, as removing
or changing the API is painful for app developers, and keeping it
ABI compatible with apps compiled against previous versions of the
library is very painful for us.

The hole idea is that generic applications shouldn't notice
if the device is using a mc-based device or not.

Regards,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Hans Verkuil]

On 03/19/2018 11:47 AM, Mauro Carvalho Chehab wrote:
> Em Mon, 19 Mar 2018 11:23:54 +0100
> Pavel Machek <pavel@ucw.cz> escreveu:
> 
>> Hi!
>>
>>> I really don't want to add functions for this to libv4l2. That's just a
>>> quick hack. The real solution is to parse this from a config
>>> file. But  
>>
>> No, this is not a quick hack. These are functions that will eventually
>> be used by the config parser. (Oh and they allow me to use camera on
>> complex hardware, but...).
>>
>> Hmm, I have mentioned that already. See quoted text below. 
>>
>>> that is a lot more work and it is something that needs to be designed
>>> properly.
>>>
>>> And that requires someone to put in the time and effort...  
>>
>> Which is what I'm trying to do. But some cooperation from your side is
>> needed, too. I acknowledged some kind of parser is needed. I can
>> do that. Are you willing to cooperate?
>>
>> But I need your feedback on the parts below. We can bikeshed about the
>> parser later.
>>
>> Do they look acceptable? Did I hook up right functions in acceptable
>> way?
>>
>> If so, yes, I can proceed with parser.
>>
>> Best regards,
>> 							Pavel
> 
> 
> Pavel,
> 
> I appreciate your efforts of adding support for mc-based devices to
> libv4l.
> 
> I guess the main poin that Hans is pointing is that we should take
> extra care in order to avoid adding new symbols to libv4l ABI/API
> without being sure that they'll be needed in long term, as removing
> or changing the API is painful for app developers, and keeping it
> ABI compatible with apps compiled against previous versions of the
> library is very painful for us.

Indeed. Sorry if I wasn't clear on that.

> The hole idea is that generic applications shouldn't notice
> if the device is using a mc-based device or not.

What is needed IMHO is an RFC that explains how you want to solve this
problem, what the parser would look like, how this would configure a
complex pipeline for use with libv4l-using applications, etc.

I.e., a full design.

And once everyone agrees that that design is solid, then it needs to be
implemented.

I really want to work with you on this, but I am not looking for partial
solutions.

I think a proper solution is a fair amount of work. If you are willing to
take that on, then that would be fantastic.

Regards,

	Hans

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2170 bytes --]

Hi!

> > Pavel,
> > 
> > I appreciate your efforts of adding support for mc-based devices to
> > libv4l.

Thanks.

> > I guess the main poin that Hans is pointing is that we should take
> > extra care in order to avoid adding new symbols to libv4l ABI/API
> > without being sure that they'll be needed in long term, as removing
> > or changing the API is painful for app developers, and keeping it
> > ABI compatible with apps compiled against previous versions of the
> > library is very painful for us.
> 
> Indeed. Sorry if I wasn't clear on that.

Aha, ok, no, I did not get that.

> > The hole idea is that generic applications shouldn't notice
> > if the device is using a mc-based device or not.
> 
> What is needed IMHO is an RFC that explains how you want to solve this
> problem, what the parser would look like, how this would configure a
> complex pipeline for use with libv4l-using applications, etc.
> 
> I.e., a full design.
> 
> And once everyone agrees that that design is solid, then it needs to be
> implemented.
> 
> I really want to work with you on this, but I am not looking for partial
> solutions.

Well, expecting design to be done for opensource development is a bit
unusual :-).

I really see two separate tasks

1) support for configuring pipeline. I believe this is best done out
of libv4l2. It outputs description file, format below. Currently I
have implemented this is in Python. File format is below.

2) support for running libv4l2 on mc-based devices. I'd like to do
that.

Description file would look like. (# comments would not be not part of file).

V4L2MEDIADESC
3 # number of files to open
/dev/video2
/dev/video6
/dev/video3
3 # number of controls to map. Controls not mentioned here go to
  # device 0 automatically. Sorted by control id.
  # Device 0 
00980913 1
009a0003 1
009a000a 2

We can parse that easily without requiring external libraries. Sorted
data allow us to do binary search.

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Hans Verkuil]

On 03/19/2018 01:00 PM, Pavel Machek wrote:
> Hi!
> 
>>> Pavel,
>>>
>>> I appreciate your efforts of adding support for mc-based devices to
>>> libv4l.
> 
> Thanks.
> 
>>> I guess the main poin that Hans is pointing is that we should take
>>> extra care in order to avoid adding new symbols to libv4l ABI/API
>>> without being sure that they'll be needed in long term, as removing
>>> or changing the API is painful for app developers, and keeping it
>>> ABI compatible with apps compiled against previous versions of the
>>> library is very painful for us.
>>
>> Indeed. Sorry if I wasn't clear on that.
> 
> Aha, ok, no, I did not get that.
> 
>>> The hole idea is that generic applications shouldn't notice
>>> if the device is using a mc-based device or not.
>>
>> What is needed IMHO is an RFC that explains how you want to solve this
>> problem, what the parser would look like, how this would configure a
>> complex pipeline for use with libv4l-using applications, etc.
>>
>> I.e., a full design.
>>
>> And once everyone agrees that that design is solid, then it needs to be
>> implemented.
>>
>> I really want to work with you on this, but I am not looking for partial
>> solutions.
> 
> Well, expecting design to be done for opensource development is a bit
> unusual :-).

Why? We have done that quite often in the past. Media is complex and you need
to decide on a design up front.

> I really see two separate tasks
> 
> 1) support for configuring pipeline. I believe this is best done out
> of libv4l2. It outputs description file, format below. Currently I
> have implemented this is in Python. File format is below.

You do need this, but why outside of libv4l2? I'm not saying I disagree
with you, but you need to give reasons for that.

> 2) support for running libv4l2 on mc-based devices. I'd like to do
> that.
> 
> Description file would look like. (# comments would not be not part of file).
> 
> V4L2MEDIADESC
> 3 # number of files to open
> /dev/video2
> /dev/video6
> /dev/video3

This won't work. The video nodes numbers (or even names) can change.
Instead these should be entity names from the media controller.

> 3 # number of controls to map. Controls not mentioned here go to
>   # device 0 automatically. Sorted by control id.
>   # Device 0 
> 00980913 1
> 009a0003 1
> 009a000a 2

You really don't need to specify the files to open. All you need is to
specify the entity ID and the list of controls that you need.

Then libv4l can just figure out which device node(s) to open for that.

> 
> We can parse that easily without requiring external libraries. Sorted
> data allow us to do binary search.

But none of this addresses setting up the initial video pipeline or
changing formats. We probably want to support that as well.

For that matter: what is it exactly that we want to support? I.e. where do
we draw the line?

A good test platform for this (outside the N900) is the i.MX6 platform.

Regards,

	Hans

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3592 bytes --]

Hi!

> >> I really want to work with you on this, but I am not looking for partial
> >> solutions.
> > 
> > Well, expecting design to be done for opensource development is a bit
> > unusual :-).
> 
> Why? We have done that quite often in the past. Media is complex and you need
> to decide on a design up front.



> > I really see two separate tasks
> > 
> > 1) support for configuring pipeline. I believe this is best done out
> > of libv4l2. It outputs description file, format below. Currently I
> > have implemented this is in Python. File format is below.
> 
> You do need this, but why outside of libv4l2? I'm not saying I disagree
> with you, but you need to give reasons for that.

I'd prefer to do this in Python. There's a lot to configure there, and
I'm not sure if libv4l2 is is right place for it. Anyway, design of 2)
does not depend on this.

> > 2) support for running libv4l2 on mc-based devices. I'd like to do
> > that.
> > 
> > Description file would look like. (# comments would not be not part of file).
> > 
> > V4L2MEDIADESC
> > 3 # number of files to open
> > /dev/video2
> > /dev/video6
> > /dev/video3
> 
> This won't work. The video nodes numbers (or even names) can change.
> Instead these should be entity names from the media controller.

Yes, it will work. 1) will configure the pipeline, and prepare
V4L2MEDIADESC file. The device names/numbers are stable after the
system is booted.

If these were entity names, v4l2_open() would have to go to /sys and
search for corresponding files... which would be rather complex and
slow.

> > 3 # number of controls to map. Controls not mentioned here go to
> >   # device 0 automatically. Sorted by control id.
> >   # Device 0 
> > 00980913 1
> > 009a0003 1
> > 009a000a 2
> 
> You really don't need to specify the files to open. All you need is to
> specify the entity ID and the list of controls that you need.
> 
> Then libv4l can just figure out which device node(s) to open for
> that.

Yes, but that would slow down v4l2_open() needlessly. I'd prefer to
avoid that.

> > We can parse that easily without requiring external libraries. Sorted
> > data allow us to do binary search.
> 
> But none of this addresses setting up the initial video pipeline or
> changing formats. We probably want to support that as well.

Well, maybe one day. But I don't believe we should attempt to support
that today.

Currently, there's no way to test that camera works on N900 with
mainline v4l2... which is rather sad. Advanced use cases can come later.

> For that matter: what is it exactly that we want to support? I.e. where do
> we draw the line?

I'd start with fixed format first. Python prepares pipeline, and
provides V4L2MEDIADESC file libv4l2 can use. You can have that this
week.

I guess it would make sense to support "application says preffered
resolution, libv4l2 attempts to set up some kind of pipeline to get
that resolution", but yes, interface there will likely be quite
complex.

Media control is more than 5 years old now. libv4l2 is still
completely uses on media control-based devices, and people are asking
for controls propagation in the kernel to fix that. My proposol
implements simple controls propagation in the userland. I believe we
should do that.

> A good test platform for this (outside the N900) is the i.MX6 platform.

Do you have one?
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Hans Verkuil]

On 03/19/2018 01:48 PM, Pavel Machek wrote:
> Hi!
> 
>>>> I really want to work with you on this, but I am not looking for partial
>>>> solutions.
>>>
>>> Well, expecting design to be done for opensource development is a bit
>>> unusual :-).
>>
>> Why? We have done that quite often in the past. Media is complex and you need
>> to decide on a design up front.
> 
> 
> 
>>> I really see two separate tasks
>>>
>>> 1) support for configuring pipeline. I believe this is best done out
>>> of libv4l2. It outputs description file, format below. Currently I
>>> have implemented this is in Python. File format is below.
>>
>> You do need this, but why outside of libv4l2? I'm not saying I disagree
>> with you, but you need to give reasons for that.
> 
> I'd prefer to do this in Python. There's a lot to configure there, and
> I'm not sure if libv4l2 is is right place for it. Anyway, design of 2)
> does not depend on this.
> 
>>> 2) support for running libv4l2 on mc-based devices. I'd like to do
>>> that.
>>>
>>> Description file would look like. (# comments would not be not part of file).
>>>
>>> V4L2MEDIADESC
>>> 3 # number of files to open
>>> /dev/video2
>>> /dev/video6
>>> /dev/video3
>>
>> This won't work. The video nodes numbers (or even names) can change.
>> Instead these should be entity names from the media controller.
> 
> Yes, it will work. 1) will configure the pipeline, and prepare
> V4L2MEDIADESC file. The device names/numbers are stable after the
> system is booted.

No, they are not. Drivers can be unloaded and reloaded, thus changing
the device nodes. The media device will give you the right major/minor
numbers and with libudev you can find the corresponding device node.
That's the right approach.

> 
> If these were entity names, v4l2_open() would have to go to /sys and
> search for corresponding files... which would be rather complex and
> slow.
> 
>>> 3 # number of controls to map. Controls not mentioned here go to
>>>   # device 0 automatically. Sorted by control id.
>>>   # Device 0 
>>> 00980913 1
>>> 009a0003 1
>>> 009a000a 2
>>
>> You really don't need to specify the files to open. All you need is to
>> specify the entity ID and the list of controls that you need.
>>
>> Then libv4l can just figure out which device node(s) to open for
>> that.
> 
> Yes, but that would slow down v4l2_open() needlessly. I'd prefer to
> avoid that.

It should be quite fast. BTW, v4l2-ctl has code to find the media device
node for a given video node.

> 
>>> We can parse that easily without requiring external libraries. Sorted
>>> data allow us to do binary search.
>>
>> But none of this addresses setting up the initial video pipeline or
>> changing formats. We probably want to support that as well.
> 
> Well, maybe one day. But I don't believe we should attempt to support
> that today.

But this should at least be thought about in the design. What is needed
in the configuration file to support this? What are the consequences for
how everything is set up? Otherwise we'd implement something only to
discover later that it doesn't work for the more advanced scenarios.

> Currently, there's no way to test that camera works on N900 with
> mainline v4l2... which is rather sad. Advanced use cases can come later.
> 
>> For that matter: what is it exactly that we want to support? I.e. where do
>> we draw the line?
> 
> I'd start with fixed format first. Python prepares pipeline, and
> provides V4L2MEDIADESC file libv4l2 can use. You can have that this
> week.

I'm not sure I want python. An embedded system might not have python
installed. Also, if we need to parse the configuration file in libv4l
(and I am 90% certain that that is what we need to do), then you don't
want to call a python script from there.

> 
> I guess it would make sense to support "application says preffered
> resolution, libv4l2 attempts to set up some kind of pipeline to get
> that resolution", but yes, interface there will likely be quite
> complex.
> 
> Media control is more than 5 years old now. libv4l2 is still
> completely uses on media control-based devices, and people are asking
> for controls propagation in the kernel to fix that. My proposol
> implements simple controls propagation in the userland. I believe we
> should do that.

Your proposal implements one specific use-case. One piece of a larger puzzle.

We first need a proper design proposal and we can go from there.

Now, I admit, all this takes time. And that's why this still hasn't been done
after all those years.

> 
>> A good test platform for this (outside the N900) is the i.MX6 platform.
> 
> Do you have one?

Yes. Although I would need to set it up, I still haven't done that.

But Steve Longerbeam is probably the right person to test this for you.

What I have been thinking of (although never in any real detail) is that we
probably want to have an application that is run from udev when a media device
is created and that reads a config file and does an initial pipeline configuration.

So once that's setup you should be able to do: 'v4l2-ctl --stream-mmap' and
get video.

Next, libv4l will also read the same configuration file and use it to provide
a compatibility layer so applications that use libv4l will work better with
MC devices. Part of that is indeed control handling.

Regards,

	Hans

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3535 bytes --]

Hi!

> >>> V4L2MEDIADESC
> >>> 3 # number of files to open
> >>> /dev/video2
> >>> /dev/video6
> >>> /dev/video3
> >>
> >> This won't work. The video nodes numbers (or even names) can change.
> >> Instead these should be entity names from the media controller.
> > 
> > Yes, it will work. 1) will configure the pipeline, and prepare
> > V4L2MEDIADESC file. The device names/numbers are stable after the
> > system is booted.
> 
> No, they are not. Drivers can be unloaded and reloaded, thus changing
> the device nodes. The media device will give you the right major/minor
> numbers and with libudev you can find the corresponding device node.
> That's the right approach.

Well, yes, drivers can be unloaded and reloaded. But at that point
pipeline will need to be set up again, so we'll get new V4L2MEDIADESC
file.

> > Yes, but that would slow down v4l2_open() needlessly. I'd prefer to
> > avoid that.
> 
> It should be quite fast. BTW, v4l2-ctl has code to find the media device
> node for a given video node.

Ok, I'll take a look, but I'd still prefer fast & simple.

> >> For that matter: what is it exactly that we want to support? I.e. where do
> >> we draw the line?
> > 
> > I'd start with fixed format first. Python prepares pipeline, and
> > provides V4L2MEDIADESC file libv4l2 can use. You can have that this
> > week.
> 
> I'm not sure I want python. An embedded system might not have python
> installed. Also, if we need to parse the configuration file in libv4l
> (and I am 90% certain that that is what we need to do), then you don't
> want to call a python script from there.

No, I don't propose calling python from libv4l2.

I propose "separate component configures the pipeline, and writes
V4L2MEDIADESC file libv4l2 then uses to map controls to devices". For
me, that component is currently python. In embededded world, I guess
they could hard-code the config, or write it in whatever language they
prefer.

> > Media control is more than 5 years old now. libv4l2 is still
> > completely uses on media control-based devices, and people are asking
> > for controls propagation in the kernel to fix that. My proposol
> > implements simple controls propagation in the userland. I believe we
> > should do that.
> 
> Your proposal implements one specific use-case. One piece of a
> larger puzzle.

Well, rather important piece, because it allows the complex cameras to
be used at all.

> >> A good test platform for this (outside the N900) is the i.MX6 platform.
> > 
> > Do you have one?
> 
> Yes. Although I would need to set it up, I still haven't done that.

Ok.

> But Steve Longerbeam is probably the right person to test this for you.
> 
> What I have been thinking of (although never in any real detail) is that we
> probably want to have an application that is run from udev when a media device
> is created and that reads a config file and does an initial pipeline configuration.
> 
> So once that's setup you should be able to do: 'v4l2-ctl --stream-mmap' and
> get video.
> 
> Next, libv4l will also read the same configuration file and use it to provide
> a compatibility layer so applications that use libv4l will work better with
> MC devices. Part of that is indeed control handling.

Yep, that would be nice. And yes, control handling is important for
me.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Mon, 19 Mar 2018 13:48:55 +0100
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > >> I really want to work with you on this, but I am not looking for partial
> > >> solutions.  
> > > 
> > > Well, expecting design to be done for opensource development is a bit
> > > unusual :-).  
> > 
> > Why? We have done that quite often in the past. Media is complex and you need
> > to decide on a design up front.  
> 
> 
> 
> > > I really see two separate tasks
> > > 
> > > 1) support for configuring pipeline. I believe this is best done out
> > > of libv4l2. It outputs description file, format below. Currently I
> > > have implemented this is in Python. File format is below.  
> > 
> > You do need this, but why outside of libv4l2? I'm not saying I disagree
> > with you, but you need to give reasons for that.  
> 
> I'd prefer to do this in Python. There's a lot to configure there, and
> I'm not sure if libv4l2 is is right place for it. Anyway, design of 2)
> does not depend on this.

Yes, the functionality of parsing the pipeline can be done later,
but, IMO, it should be integrated at the library.

> > > 2) support for running libv4l2 on mc-based devices. I'd like to do
> > > that.
> > > 
> > > Description file would look like. (# comments would not be not part of file).
> > > 
> > > V4L2MEDIADESC
> > > 3 # number of files to open
> > > /dev/video2
> > > /dev/video6
> > > /dev/video3  
> > 
> > This won't work. The video nodes numbers (or even names) can change.
> > Instead these should be entity names from the media controller.  
> 
> Yes, it will work. 1) will configure the pipeline, and prepare
> V4L2MEDIADESC file. The device names/numbers are stable after the
> system is booted.

No, it doensn't. On more complex SoC chips (like Exynos) with multiple
drivers for video capture, mpeg encoding/decoding, etc, the device node
numbers change on every reboot, as they're probed by different drivers.
So, it depends on what device driver is probed first.

That's a list of such devices on an Exynos 5 hardware:

 $ v4l2-ctl --list-devices
 s5p-mfc-dec (platform:11000000.codec):
 	/dev/video4
 	/dev/video5
 
 s5p-jpeg encoder (platform:11f50000.jpeg):
 	/dev/video0
 	/dev/video1
 
 s5p-jpeg encoder (platform:11f60000.jpeg):
 	/dev/video2
 	/dev/video3
 
 exynos-gsc gscaler (platform:13e00000.video-scaler):
 	/dev/video6
 
 exynos-gsc gscaler (platform:13e10000.video-scaler):
 	/dev/video7
 

(on every boot, the /dev/video? numbe changes).

We need to use something using sysfs in order to get it right.
That's what I did on some test scripts here:

	#!/bin/bash
	NEEDED=platform-11000000.codec-video-index0
	DEV=$(ls -l /dev/v4l/by-path/$NEEDED|perl -ne ' print $1 if (m,/video(\d+),)')
	echo "jpeg scaler ($NEEDED) at: /dev/video$DEV"
	gst-launch-1.0 filesrc location=~/test.mov ! qtdemux ! h264parse ! v4l2video${DEV}dec ! videoconvert ! kmssink

But that's because I was just too lazy to do the right thing (and this was
for a test machine - so no real need for it to be stable enough).


> If these were entity names, v4l2_open() would have to go to /sys and
> search for corresponding files... which would be rather complex and
> slow.

It has to, but speed here is not the problem, as this happens only
at open() time, and sysfs parsing is fast, as it is not stored on
disk.

> 
> > > We can parse that easily without requiring external libraries. Sorted
> > > data allow us to do binary search.  
> > 
> > But none of this addresses setting up the initial video pipeline or
> > changing formats. We probably want to support that as well.  
> 
> Well, maybe one day. But I don't believe we should attempt to support
> that today.
> 
> Currently, there's no way to test that camera works on N900 with
> mainline v4l2... which is rather sad. Advanced use cases can come later.
> 
> > For that matter: what is it exactly that we want to support? I.e. where do
> > we draw the line?  
> 
> I'd start with fixed format first. Python prepares pipeline, and
> provides V4L2MEDIADESC file libv4l2 can use. You can have that this
> week.

I don't see any problems with starting implementing it for controls,
but we need to have a broad view when designing, in order to avoid
breaking APIs and file formats.

> I guess it would make sense to support "application says preffered
> resolution, libv4l2 attempts to set up some kind of pipeline to get
> that resolution", but yes, interface there will likely be quite
> complex.

I don't it will be that complex. A simple pipeline setting like what
I proposed on the previous email:

	[pipeline pipe1]
		link0 = SGRBG10 640x480: entity1:0 -> entity2:0[1]
		link1 = SGRBG10 640x480: entity2:2-> entity3:0[1]
		link2 = UYVY 640x480: entity3:1-> entity4:0[1]
		link3 = UYVY 640x480: entity4:1-> entity5:0[1]

		sink0 = UYVY 320x200: entity5:0[1]
		sink1 = UYVY 640x480: entity3:0[1]

Seems enough to let the library to know what to do if an app wants to
get a pipeline with UYVY output at 320x200 or at 640x480 resolutions.

Thanks,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Mon, 19 Mar 2018 13:15:22 +0100
Hans Verkuil <hverkuil@xs4all.nl> escreveu:

> On 03/19/2018 01:00 PM, Pavel Machek wrote:
> > Hi!
> >   
> >>> Pavel,
> >>>
> >>> I appreciate your efforts of adding support for mc-based devices to
> >>> libv4l.  
> > 
> > Thanks.
> >   
> >>> I guess the main poin that Hans is pointing is that we should take
> >>> extra care in order to avoid adding new symbols to libv4l ABI/API
> >>> without being sure that they'll be needed in long term, as removing
> >>> or changing the API is painful for app developers, and keeping it
> >>> ABI compatible with apps compiled against previous versions of the
> >>> library is very painful for us.  
> >>
> >> Indeed. Sorry if I wasn't clear on that.  
> > 
> > Aha, ok, no, I did not get that.
> >   
> >>> The hole idea is that generic applications shouldn't notice
> >>> if the device is using a mc-based device or not.  
> >>
> >> What is needed IMHO is an RFC that explains how you want to solve this
> >> problem, what the parser would look like, how this would configure a
> >> complex pipeline for use with libv4l-using applications, etc.
> >>
> >> I.e., a full design.
> >>
> >> And once everyone agrees that that design is solid, then it needs to be
> >> implemented.
> >>
> >> I really want to work with you on this, but I am not looking for partial
> >> solutions.  
> > 
> > Well, expecting design to be done for opensource development is a bit
> > unusual :-).  
> 
> Why? We have done that quite often in the past. Media is complex and you need
> to decide on a design up front.
> 
> > I really see two separate tasks
> > 
> > 1) support for configuring pipeline. I believe this is best done out
> > of libv4l2. It outputs description file, format below. Currently I
> > have implemented this is in Python. File format is below.  
> 
> You do need this, but why outside of libv4l2? I'm not saying I disagree
> with you, but you need to give reasons for that.
> 
> > 2) support for running libv4l2 on mc-based devices. I'd like to do
> > that.
> > 
> > Description file would look like. (# comments would not be not part of file).
> > 
> > V4L2MEDIADESC
> > 3 # number of files to open
> > /dev/video2
> > /dev/video6
> > /dev/video3  

"Easy" file formats usually means maintenance troubles as new
things are needed, and makes worse for users to write it. 
You should take a look at lib/libdvbv5/dvb-file.c, mainly at 
fill_entry() and dvb_read_file().

It has a parser there for things like:

[foo]
	bar = value

and it doesn't require external libraries. I considered writing it
as a small helper function, but it turns that parsing this format
is so simple that can easily be added anywhere else.

The advantage of such format is that it should be simple enough
to add more stuff there, and intuitive enough for end users.

> This won't work. The video nodes numbers (or even names) can change.

Yes. That all depends on how udev is set on a given system.
Btw, the logic should likely use libudev in order to get the
devname.

> Instead these should be entity names from the media controller.

Agreed that it should use MC. Yet, IMHO, the best would be to use
the entity function instead, as entity names might eventually
change on newer versions of the Kernel.

So, IMHO, entities should be described as:

	[entity entity1]
		name = foo
		function = bar

(again, user may specify just name, just function or both)

> > 3 # number of controls to map. Controls not mentioned here go to
> >   # device 0 automatically. Sorted by control id.
> >   # Device 0 
> > 00980913 1
> > 009a0003 1
> > 009a000a 2  

I would, instead, encode it as:

	[control white balance]
		control_id = 0x00980913
		entity = foo_entity_name

	[control exposure]
		control_id = V4L2_CID_EXPOSURE_ABSOLUTE
		entity = bar_entity_name
	...

Allowing both hexadecimal values and control macro names (can easily parsed 
from the header file, as we already do for other things with "make sync").

Makes a way easier for people to understand what that means and to write
such files.

> You really don't need to specify the files to open. All you need is to
> specify the entity ID and the list of controls that you need.
> 
> Then libv4l can just figure out which device node(s) to open for that.

The only file it could make sense to specify is the media
controller device or the name of the media controller device,
as more than one /dev/media? devnode could be present on a given
system.

I would actually support both, e. g.:

[media controller]

	name = foo
	devname = bar

Only "name" or "devname" would be required:
	- If "name" is blank, it would just use "devname";
	- If "devname" is blank, it would open all /dev/media?
	  devices until it matches "name";
	- If both are given, it would open "devname" and check
	  if "name" matches the media controller name, failing
	  otherwise.

> > 
> > We can parse that easily without requiring external libraries. Sorted
> > data allow us to do binary search.  
> 
> But none of this addresses setting up the initial video pipeline or
> changing formats. We probably want to support that as well.

It should probably be easy to add a generic pipeline descriptor
with a format like:

	[pipeline pipe1]
		link0 = SGRBG10 640x480: entity1:0 -> entity2:0[1]
		link1 = SGRBG10 640x480: entity2:2-> entity3:0[1]
		link2 = UYVY 640x480: entity3:1-> entity4:0[1]
		link3 = UYVY 640x480: entity4:1-> entity5:0[1]

		sink0 = UYVY 320x200: entity5:0[1]
		sink1 = UYVY 640x480: entity3:0[1]

> 
> For that matter: what is it exactly that we want to support? I.e. where do
> we draw the line?
> 
> A good test platform for this (outside the N900) is the i.MX6 platform.
> 
> Regards,
> 
> 	Hans



Thanks,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]

Hi!

> > > 2) support for running libv4l2 on mc-based devices. I'd like to do
> > > that.
> > > 
> > > Description file would look like. (# comments would not be not part of file).
> > > 
> > > V4L2MEDIADESC
> > > 3 # number of files to open
> > > /dev/video2
> > > /dev/video6
> > > /dev/video3  
> 
> "Easy" file formats usually means maintenance troubles as new
> things are needed, and makes worse for users to write it. 
> You should take a look at lib/libdvbv5/dvb-file.c, mainly at 
> fill_entry() and dvb_read_file().

Well, file formats just need to be maintained.

> > Instead these should be entity names from the media controller.
> 
> Agreed that it should use MC. Yet, IMHO, the best would be to use
> the entity function instead, as entity names might eventually
> change on newer versions of the Kernel.

Kernel interfaces are not supposed to be changing.

> (again, user may specify just name, just function or both)
> 
> > > 3 # number of controls to map. Controls not mentioned here go to
> > >   # device 0 automatically. Sorted by control id.
> > >   # Device 0 
> > > 00980913 1
> > > 009a0003 1
> > > 009a000a 2  
> 
> I would, instead, encode it as:
> 
> 	[control white balance]
> 		control_id = 0x00980913
> 		entity = foo_entity_name

Ok, that's really overly complex. If futrue extensibility is concern
we can do

0x00980913=whatever.

> Allowing both hexadecimal values and control macro names (can easily parsed 
> from the header file, as we already do for other things with "make
> sync").

"Easily" as in "more complex then rest of proposed code combined" :-(.

> It should probably be easy to add a generic pipeline descriptor
> with a format like:
> 
> 	[pipeline pipe1]
> 		link0 = SGRBG10 640x480: entity1:0 -> entity2:0[1]
> 		link1 = SGRBG10 640x480: entity2:2-> entity3:0[1]
> 		link2 = UYVY 640x480: entity3:1-> entity4:0[1]
> 		link3 = UYVY 640x480: entity4:1-> entity5:0[1]
> 
> 		sink0 = UYVY 320x200: entity5:0[1]
> 		sink1 = UYVY 640x480: entity3:0[1]

Well, first, this looks like very unsuitable for key=value file. Plus,
it will not be easy to parse. .... and control map needs to be
per-pipeline-configuration. Again, proposed Windows-style format can
not easily do that.

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1118 bytes --]

Hi!

> So, IMHO, entities should be described as:
> 
> 	[entity entity1]
> 		name = foo
> 		function = bar

I don't really think windows-style config file is suitable here, as we
have more than two "nested blocks".

What about something like this? Note that I'd only implement the
controls mapping for now... but it should be extensible later to setup
mappings for the application.

Best regards,
								Pavel


#modes: 2
Driver name: OMAP 3 resizer
Mode: 3000x1800
 #devices: 2
  0: et8ek8 sensor
  1: OMAP3 resizer
 #controls: 2
  0x4321a034: 1
  0x4113aab0: 1
 #links: 1
  link:
   entity1: et8ek8 sensor:1
   entity2: OMAP3 resizer:0
   resolution1: 1024x768
   resolution2: 1024x768
Mode: 1024x768
 #devices: 2
  0: et8ek8 sensor
  1: OMAP3 resizer
 #controls: 2
  0x4321a034: 1
  0x4113aab0: 1
 #links: 1
  link:
   entity1: et8ek8 sensor:1
   entity2: OMAP3 resizer:0
   resolution1: 1024x768
   resolution2: 1024x768



-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Tue, 15 May 2018 22:01:17 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > So, IMHO, entities should be described as:
> > 
> > 	[entity entity1]
> > 		name = foo
> > 		function = bar  
> 
> I don't really think windows-style config file is suitable here, as we
> have more than two "nested blocks".
> 
> What about something like this? Note that I'd only implement the
> controls mapping for now... but it should be extensible later to setup
> mappings for the application.
> 
> Best regards,
> 								Pavel
> 
> 
> #modes: 2
> Driver name: OMAP 3 resizer

It probably makes sense to place the driver name before #modes.

From what I understood, the "#foo: <number>" is a tag that makes the
parser to expect for <number> of "foo".

I would also add a first line at the beginning describing the
version of the format, just in case we add more stuff that
would require to change something at the format.

Except for that, it seems ok.

> Mode: 3000x1800
>  #devices: 2
>   0: et8ek8 sensor
>   1: OMAP3 resizer
>  #controls: 2
>   0x4321a034: 1
>   0x4113aab0: 1
>  #links: 1
>   link:
>    entity1: et8ek8 sensor:1
>    entity2: OMAP3 resizer:0
>    resolution1: 1024x768
>    resolution2: 1024x768
> Mode: 1024x768
>  #devices: 2
>   0: et8ek8 sensor
>   1: OMAP3 resizer
>  #controls: 2
>   0x4321a034: 1
>   0x4113aab0: 1
>  #links: 1
>   link:
>    entity1: et8ek8 sensor:1
>    entity2: OMAP3 resizer:0
>    resolution1: 1024x768
>    resolution2: 1024x768
> 
> 
> 



Thanks,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 13574 bytes --]

Hi!

> > #modes: 2
> > Driver name: OMAP 3 resizer
> 
> It probably makes sense to place the driver name before #modes.

I dropped the driver for now. It served no purpose...

I got the patches to work on N900. With them (and right)

> From what I understood, the "#foo: <number>" is a tag that makes the
> parser to expect for <number> of "foo".
> 
> I would also add a first line at the beginning describing the
> version of the format, just in case we add more stuff that
> would require to change something at the format.

Ok, done.

> Except for that, it seems ok.

Good, thanks!

Here's current version of the patch. Code will need moving to
libv4l2. Would "v4l2_open_pipeline()" be suitable name to use?

Best regards,
								Pavel

diff --git a/contrib/test/sdlcam.c b/contrib/test/sdlcam.c
index cc43a10..95f85f9 100644
--- a/contrib/test/sdlcam.c
+++ b/contrib/test/sdlcam.c
@@ -8,7 +8,7 @@
    Copyright 2017 Pavel Machek, LGPL
 
    Needs sdl2, sdl2_image libraries. sudo aptitude install libsdl2-dev
-   libsdl2-image-dev on Debian systems.
+   libsdl2-image-dev libjpeg-dev on Debian systems.
 */
 
 #include <time.h>
@@ -26,6 +26,7 @@
 #include <sys/time.h>
 #include <sys/ioctl.h>
 #include <fcntl.h>
+#include <dirent.h>
 
 #include <jpeglib.h>
 
@@ -1172,11 +1173,18 @@ static void sdl_iteration(struct sdl *m)
 	}
 }
 
+static int open_complex(char *name, int v4l2_flags);
+
 static void cam_open(struct dev_info *dev, char *name)
 {
 	struct v4l2_format *fmt = &dev->fmt;
 
-	dev->fd = v4l2_open(name, O_RDWR);
+	//	dev->fd = v4l2_open(name, O_RDWR);
+	//dev->fd = open_complex("/my/tui/camera/n900.cv", 0);
+	dev->fd = open_complex("/data/pavel/g/v4l-utils/test.txt", 0);
+	//dev->fd = open(name, O_RDWR);
+	//printf("fd = %d\n", dev->fd);
+	//dev->fd = v4l2_fd_open(dev->fd, 0);
 	if (dev->fd < 0) {
 		printf("video %s open failed: %m\n", name);
 		exit(1);
@@ -1198,6 +1206,179 @@ static void cam_open(struct dev_info *dev, char *name)
 
 /* ------------------------------------------------------------------ */
 
+static void scan_devices(char **device_names, int *device_fds, int num)
+{
+	struct dirent **namelist;
+	int n;
+	char *class_v4l = "/sys/class/video4linux";
+
+	n = scandir(class_v4l, &namelist, NULL, alphasort);
+	if (n < 0) {
+		perror("scandir");
+		return;
+	}
+	
+	while (n--) {
+		if (namelist[n]->d_name[0] != '.') {
+			char filename[1024], content[1024];
+			sprintf(filename, "%s/%s/name", class_v4l, namelist[n]->d_name);
+			FILE *f = fopen(filename, "r");
+			if (!f) {
+				printf("Strange, can't open %s", filename);
+			} else {
+				fgets(content, 1024, f);
+				fclose(f);
+				printf("device %s : %s\n", namelist[n]->d_name, content);
+				int i;
+				for (i = num-1; i >=0; i--) {
+					if (!strcmp(content, device_names[i])) {
+						sprintf(filename, "/dev/%s", namelist[n]->d_name);
+						device_fds[i] = open(filename, O_RDWR);
+						if (device_fds[i] < 0) {
+							printf("Error opening %s: %m\n", filename);
+						}
+						printf("*** found match - %s %d\n", filename, device_fds[i]);
+					}
+				}
+			}
+		}
+		free(namelist[n]);
+	}
+	free(namelist);
+  
+}
+
+static int open_complex(char *name, int v4l2_flags)
+{
+#define perr(s) printf("v4l2: open_complex: " s "\n");
+#define BUF 256
+	FILE *f = fopen(name, "r");
+
+	int res = -1;
+	char buf[BUF];
+	int version, num_modes, num_devices, num_controls;
+	int dev, control;
+
+	if (!f) {
+		perr("open of .cv file failed: %m");
+		goto err;
+	}
+
+	if (fscanf(f, "Complex Video: %d\n", &version) != 1) {
+		perr(".cv file does not have required header");
+		goto close;
+	}
+
+	if (version != 0) {
+		perr(".cv file has unknown version");
+		goto close;
+	}
+  
+	if (fscanf(f, "#modes: %d\n", &num_modes) != 1) {
+		perr("could not parse modes");
+		goto close;
+	}
+
+	if (num_modes != 1) {
+		perr("only single mode is supported for now");
+		goto close;
+	}
+
+	if (fscanf(f, "Mode: %s\n", buf) != 1) {
+		perr("could not parse mode name");
+		goto close;
+	}
+
+	if (fscanf(f, " #devices: %d\n", &num_devices) != 1) {
+		perr("could not parse number of devices");
+		goto close;
+	}
+#define MAX_DEVICES 16
+	char *device_names[MAX_DEVICES] = { NULL, };
+	int device_fds[MAX_DEVICES];
+	if (num_devices > MAX_DEVICES) {
+		perr("too many devices");
+		goto close;
+	}
+  
+	for (dev = 0; dev < num_devices; dev++) {
+		int tmp;
+		if (fscanf(f, "%d: ", &tmp) != 1) {
+			perr("could not parse device");
+			goto free_devices;
+		}
+		if (tmp != dev) {
+			perr("bad device number");
+			goto free_devices;
+		}
+		fgets(buf, BUF, f);
+		printf("Device %d %d %s\n", dev, tmp, buf);
+		device_names[dev] = strdup(buf);
+		device_fds[dev] = -1;
+	}
+
+	scan_devices(device_names, device_fds, num_devices);
+
+	for (dev = 0; dev < num_devices; dev++) {
+		printf("Device %d fd %d\n", dev, device_fds[dev]);
+		if (device_fds[dev] == -1) {
+			perr("Could not open all required devices");
+			goto close_devices;
+		}
+	}
+
+	if (fscanf(f, " #controls: %d\n", &num_controls) != 1) {
+		perr("can not parse number of controls");
+		goto close_devices;
+	}
+
+	struct v4l2_controls_map *map = malloc(sizeof(struct v4l2_controls_map) +
+					       num_controls*sizeof(struct v4l2_control_map));
+
+	map->num_controls = num_controls;
+	map->num_fds = num_devices;
+	map->main_fd = device_fds[0];
+  
+	for (control = 0; control < num_controls; control++) {
+		unsigned long num;
+		int dev;
+		if (fscanf(f, "0x%lx: %d\n", &num, &dev) != 2) {
+			perr("could not parse control");
+			goto free_map;
+		}
+		if ((dev < 0) || (dev >= num_devices)) {
+			perr("device out of range");
+			goto free_map;
+		}
+		map->map[control].control = num;
+		map->map[control].fd = device_fds[dev];
+		printf("---> map: %lx %d\n", num, device_fds[dev]);
+	}
+	if (fscanf(f, "%s", buf) > 0) {
+		perr("junk at end of file");
+		goto free_map;
+	}
+
+	printf("Success, main fd is %d\n", map->main_fd);
+	res = v4l2_open_pipeline(map, v4l2_flags);
+
+	if (res < 0) {
+free_map:
+		free(map);
+close_devices:
+		for (dev = 0; dev < num_devices; dev++)
+			close(device_fds[dev]);
+	}
+free_devices:
+	for (dev = 0; dev < num_devices; dev++) {
+		free(device_names[dev]);
+	}
+close:
+	fclose(f);
+err:
+	return res;
+}
+
 
 static struct dev_info dev;
 
@@ -1206,6 +1387,8 @@ int main(void)
 	int i;
 	struct v4l2_format *fmt = &dev.fmt;
 
+	//open_complex("/my/tui/camera/n900.cv");
+
 	dtime();
 	cam_open(&dev, "/dev/video0");
 
diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
index ea1870d..6220dfd 100644
--- a/lib/include/libv4l2.h
+++ b/lib/include/libv4l2.h
@@ -109,6 +109,23 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
    (note the fd is left open in this case). */
 LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
 
+struct v4l2_control_map {
+	unsigned long control;
+	int fd;
+};
+
+struct v4l2_controls_map {
+	int main_fd;
+	int num_fds;
+	int num_controls;
+	struct v4l2_control_map map[];
+};
+
+LIBV4L_PUBLIC int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
+
+LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
+
+
 #ifdef __cplusplus
 }
 #endif /* __cplusplus */
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 1924c91..ebe5dad 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -104,6 +104,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+	struct v4l2_controls_map *map;
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 2db25d1..6bb232d 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -787,6 +787,8 @@ no_capture:
 	if (index >= devices_used)
 		devices_used = index + 1;
 
+	devices[index].map = NULL;
+
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
 	   a frame rate using the S_PARM ioctl after a S_FMT */
@@ -1056,12 +1058,47 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+int v4l2_get_fd_for_control(int fd, unsigned long control)
+{
+	int index = v4l2_get_index(fd);
+	struct v4l2_controls_map *map;
+	int lo = 0;
+	int hi;
+
+	if (index < 0)
+		return fd;
+
+	map = devices[index].map;
+	if (!map)
+		return fd;
+	hi = map->num_controls;
+
+	while (lo < hi) {
+		int i = (lo + hi) / 2;
+		if (map->map[i].control == control) {
+			return map->map[i].fd;
+		}
+		if (map->map[i].control > control) {
+			hi = i;
+			continue;
+		}
+		if (map->map[i].control < control) {
+			lo = i+1;
+			continue;
+		}
+		printf("Bad: impossible condition in binary search\n");
+		exit(1);
+	}
+	return fd;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
 	va_list ap;
 	int result, index, saved_err;
-	int is_capture_request = 0, stream_needs_locking = 0;
+	int is_capture_request = 0, stream_needs_locking = 0, 
+	    is_subdev_request = 0;
 
 	va_start(ap, request);
 	arg = va_arg(ap, void *);
@@ -1076,18 +1113,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	   ioctl, causing it to get sign extended, depending upon this behavior */
 	request = (unsigned int)request;
 
+	/* FIXME */
 	if (devices[index].convert == NULL)
 		goto no_capture_request;
 
 	/* Is this a capture request and do we need to take the stream lock? */
 	switch (request) {
-	case VIDIOC_QUERYCAP:
 	case VIDIOC_QUERYCTRL:
 	case VIDIOC_G_CTRL:
 	case VIDIOC_S_CTRL:
 	case VIDIOC_G_EXT_CTRLS:
-	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_S_EXT_CTRLS:
+		is_subdev_request = 1;
+	case VIDIOC_QUERYCAP:
+	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_ENUM_FRAMESIZES:
 	case VIDIOC_ENUM_FRAMEINTERVALS:
 		is_capture_request = 1;
@@ -1151,10 +1190,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	}
 
 	if (!is_capture_request) {
+	  int sub_fd;
 no_capture_request:
+		  sub_fd = fd;
+		if (is_subdev_request) {
+		  sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
+		}
 		result = devices[index].dev_ops->ioctl(
 				devices[index].dev_ops_priv,
-				fd, request, arg);
+				sub_fd, request, arg);
 		saved_err = errno;
 		v4l2_log_ioctl(request, arg, result);
 		errno = saved_err;
@@ -1782,3 +1826,28 @@ int v4l2_get_control(int fd, int cid)
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }
+
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
+{
+	int index;
+	int i;
+
+	for (i=0; i<map->num_controls; i++) {
+	  printf("%lx %d\n", map->map[i].control, map->map[i].fd);
+	  if (map->map[i].fd <= 0) {
+	    printf("Bad fd in map\n");
+	    return -1;
+	  }
+	  if (i>=1 && map->map[i].control <= map->map[i-1].control) {
+	    printf("Not sorted\n");
+	    return -1;
+	  }
+	}
+
+	i = v4l2_fd_open(map->main_fd, v4l2_flags);
+	index = v4l2_get_index(map->main_fd);
+	devices[index].map = map;
+	return i;
+}
+
diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
index 59f28b1..c1e6f93 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.c
+++ b/lib/libv4lconvert/control/libv4lcontrol.c
@@ -863,6 +863,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 	struct v4l2_queryctrl *ctrl = arg;
 	int retval;
 	uint32_t orig_id = ctrl->id;
+	int fd;
 
 	/* if we have an exact match return it */
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
@@ -872,8 +873,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
 	/* find out what the kernel driver would respond. */
-	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_QUERYCTRL, arg);
 
 	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
@@ -903,6 +905,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -911,7 +914,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_G_CTRL, arg);
 }
 
@@ -994,6 +998,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -1008,7 +1013,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_S_CTRL, arg);
 }
 


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 11769 bytes --]

Hi!

Ok, can I get any comments on this one?
v4l2_open_complex("/file/with/descriptor", 0) can be used to open
whole pipeline at once, and work if it as if it was one device.

Thanks,
								Pavel

diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
index ea1870d..f170c3d 100644
--- a/lib/include/libv4l2.h
+++ b/lib/include/libv4l2.h
@@ -109,6 +109,12 @@ LIBV4L_PUBLIC int v4l2_get_control(int fd, int cid);
    (note the fd is left open in this case). */
 LIBV4L_PUBLIC int v4l2_fd_open(int fd, int v4l2_flags);
 
+/* v4l2_open_complex: open complex pipeline. Name of pipeline descriptor
+   is passed to v4l2_open_complex(). It opens devices described there and
+   handles mapping controls between devices. 
+ */  
+LIBV4L_PUBLIC int v4l2_open_complex(char *name, int v4l2_flags);
+
 #ifdef __cplusplus
 }
 #endif /* __cplusplus */
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 1924c91..1ee697a 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -104,6 +104,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+	struct v4l2_controls_map *map;
 };
 
 /* From v4l2-plugin.c */
@@ -130,4 +131,20 @@ static inline void v4l2_plugin_cleanup(void *plugin_lib, void *plugin_priv,
 extern const char *v4l2_ioctls[];
 void v4l2_log_ioctl(unsigned long int request, void *arg, int result);
 
+
+struct v4l2_control_map {
+	unsigned long control;
+	int fd;
+};
+
+struct v4l2_controls_map {
+	int main_fd;
+	int num_fds;
+	int num_controls;
+	struct v4l2_control_map map[];
+};
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
+LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
+
 #endif
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 2db25d1..39f69db 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -70,6 +70,8 @@
 #include <sys/types.h>
 #include <sys/mman.h>
 #include <sys/stat.h>
+#include <dirent.h>
+
 #include "libv4l2.h"
 #include "libv4l2-priv.h"
 #include "libv4l-plugin.h"
@@ -787,6 +789,8 @@ no_capture:
 	if (index >= devices_used)
 		devices_used = index + 1;
 
+	devices[index].map = NULL;
+
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
 	   a frame rate using the S_PARM ioctl after a S_FMT */
@@ -1056,12 +1060,47 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+int v4l2_get_fd_for_control(int fd, unsigned long control)
+{
+	int index = v4l2_get_index(fd);
+	struct v4l2_controls_map *map;
+	int lo = 0;
+	int hi;
+
+	if (index < 0)
+		return fd;
+
+	map = devices[index].map;
+	if (!map)
+		return fd;
+	hi = map->num_controls;
+
+	while (lo < hi) {
+		int i = (lo + hi) / 2;
+		if (map->map[i].control == control) {
+			return map->map[i].fd;
+		}
+		if (map->map[i].control > control) {
+			hi = i;
+			continue;
+		}
+		if (map->map[i].control < control) {
+			lo = i+1;
+			continue;
+		}
+		printf("Bad: impossible condition in binary search\n");
+		exit(1);
+	}
+	return fd;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
 	va_list ap;
 	int result, index, saved_err;
-	int is_capture_request = 0, stream_needs_locking = 0;
+	int is_capture_request = 0, stream_needs_locking = 0, 
+	    is_subdev_request = 0;
 
 	va_start(ap, request);
 	arg = va_arg(ap, void *);
@@ -1076,18 +1115,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	   ioctl, causing it to get sign extended, depending upon this behavior */
 	request = (unsigned int)request;
 
+	/* FIXME */
 	if (devices[index].convert == NULL)
 		goto no_capture_request;
 
 	/* Is this a capture request and do we need to take the stream lock? */
 	switch (request) {
-	case VIDIOC_QUERYCAP:
 	case VIDIOC_QUERYCTRL:
 	case VIDIOC_G_CTRL:
 	case VIDIOC_S_CTRL:
 	case VIDIOC_G_EXT_CTRLS:
-	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_S_EXT_CTRLS:
+		is_subdev_request = 1;
+	case VIDIOC_QUERYCAP:
+	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_ENUM_FRAMESIZES:
 	case VIDIOC_ENUM_FRAMEINTERVALS:
 		is_capture_request = 1;
@@ -1151,10 +1192,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	}
 
 	if (!is_capture_request) {
+	  int sub_fd;
 no_capture_request:
+		  sub_fd = fd;
+		if (is_subdev_request) {
+		  sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
+		}
 		result = devices[index].dev_ops->ioctl(
 				devices[index].dev_ops_priv,
-				fd, request, arg);
+				sub_fd, request, arg);
 		saved_err = errno;
 		v4l2_log_ioctl(request, arg, result);
 		errno = saved_err;
@@ -1782,3 +1828,195 @@ int v4l2_get_control(int fd, int cid)
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }
+
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
+{
+	int index;
+	int i;
+
+	for (i=0; i<map->num_controls; i++) {
+		if (map->map[i].fd <= 0) {
+			V4L2_LOG_ERR("v4l2_open_pipeline: Bad fd in map.\n");
+			return -1;
+		}
+		if (i>=1 && map->map[i].control <= map->map[i-1].control) {
+			V4L2_LOG_ERR("v4l2_open_pipeline: Controls not sorted.\n");
+			return -1;
+		}
+	}
+
+	i = v4l2_fd_open(map->main_fd, v4l2_flags);
+	index = v4l2_get_index(map->main_fd);
+	devices[index].map = map;
+	return i;
+}
+
+static void scan_devices(char **device_names, int *device_fds, int num)
+{
+	struct dirent **namelist;
+	int n;
+	char *class_v4l = "/sys/class/video4linux";
+
+	n = scandir(class_v4l, &namelist, NULL, alphasort);
+	if (n < 0) {
+		perror("scandir");
+		return;
+	}
+	
+	while (n--) {
+		if (namelist[n]->d_name[0] != '.') {
+			char filename[1024], content[1024];
+			sprintf(filename, "%s/%s/name", class_v4l, namelist[n]->d_name);
+			FILE *f = fopen(filename, "r");
+			if (!f) {
+				printf("Strange, can't open %s", filename);
+			} else {
+				fgets(content, 1024, f);
+				fclose(f);
+
+				int i;
+				for (i = num-1; i >=0; i--) {
+					if (!strcmp(content, device_names[i])) {
+						sprintf(filename, "/dev/%s", namelist[n]->d_name);
+						device_fds[i] = open(filename, O_RDWR);
+						if (device_fds[i] < 0) {
+							V4L2_LOG_ERR("Error opening %s: %m\n", filename);
+						}
+					}
+				}
+			}
+		}
+		free(namelist[n]);
+	}
+	free(namelist);
+  
+}
+
+int v4l2_open_complex(char *name, int v4l2_flags)
+{
+#define perr(s) V4L2_LOG_ERR("open_complex: " s "\n")
+#define BUF 256
+	FILE *f = fopen(name, "r");
+
+	int res = -1;
+	char buf[BUF];
+	int version, num_modes, num_devices, num_controls;
+	int dev, control;
+
+	if (!f) {
+		perr("open of .cv file failed: %m");
+		goto err;
+	}
+
+	if (fscanf(f, "Complex Video: %d\n", &version) != 1) {
+		perr(".cv file does not have required header");
+		goto close;
+	}
+
+	if (version != 0) {
+		perr(".cv file has unknown version");
+		goto close;
+	}
+  
+	if (fscanf(f, "#modes: %d\n", &num_modes) != 1) {
+		perr("could not parse modes");
+		goto close;
+	}
+
+	if (num_modes != 1) {
+		perr("only single mode is supported for now");
+		goto close;
+	}
+
+	if (fscanf(f, "Mode: %s\n", buf) != 1) {
+		perr("could not parse mode name");
+		goto close;
+	}
+
+	if (fscanf(f, " #devices: %d\n", &num_devices) != 1) {
+		perr("could not parse number of devices");
+		goto close;
+	}
+#define MAX_DEVICES 16
+	char *device_names[MAX_DEVICES] = { NULL, };
+	int device_fds[MAX_DEVICES];
+	if (num_devices > MAX_DEVICES) {
+		perr("too many devices");
+		goto close;
+	}
+  
+	for (dev = 0; dev < num_devices; dev++) {
+		int tmp;
+		if (fscanf(f, "%d: ", &tmp) != 1) {
+			perr("could not parse device");
+			goto free_devices;
+		}
+		if (tmp != dev) {
+			perr("bad device number");
+			goto free_devices;
+		}
+		fgets(buf, BUF, f);
+		device_names[dev] = strdup(buf);
+		device_fds[dev] = -1;
+	}
+
+	scan_devices(device_names, device_fds, num_devices);
+
+	for (dev = 0; dev < num_devices; dev++) {
+		if (device_fds[dev] == -1) {
+			perr("Could not open all required devices");
+			goto close_devices;
+		}
+	}
+
+	if (fscanf(f, " #controls: %d\n", &num_controls) != 1) {
+		perr("can not parse number of controls");
+		goto close_devices;
+	}
+
+	struct v4l2_controls_map *map = malloc(sizeof(struct v4l2_controls_map) +
+					       num_controls*sizeof(struct v4l2_control_map));
+
+	map->num_controls = num_controls;
+	map->num_fds = num_devices;
+	map->main_fd = device_fds[0];
+  
+	for (control = 0; control < num_controls; control++) {
+		unsigned long num;
+		int dev;
+		if (fscanf(f, "0x%lx: %d\n", &num, &dev) != 2) {
+			perr("could not parse control");
+			goto free_map;
+		}
+		if ((dev < 0) || (dev >= num_devices)) {
+			perr("device out of range");
+			goto free_map;
+		}
+		map->map[control].control = num;
+		map->map[control].fd = device_fds[dev];
+	}
+	if (fscanf(f, "%s", buf) > 0) {
+		perr("junk at end of file");
+		goto free_map;
+	}
+
+	res = v4l2_open_pipeline(map, v4l2_flags);
+
+	if (res < 0) {
+free_map:
+		free(map);
+close_devices:
+		for (dev = 0; dev < num_devices; dev++)
+			close(device_fds[dev]);
+	}
+free_devices:
+	for (dev = 0; dev < num_devices; dev++) {
+		free(device_names[dev]);
+	}
+close:
+	fclose(f);
+err:
+	return res;
+}
+
diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
index 59f28b1..c1e6f93 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.c
+++ b/lib/libv4lconvert/control/libv4lcontrol.c
@@ -863,6 +863,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 	struct v4l2_queryctrl *ctrl = arg;
 	int retval;
 	uint32_t orig_id = ctrl->id;
+	int fd;
 
 	/* if we have an exact match return it */
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
@@ -872,8 +873,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
 	/* find out what the kernel driver would respond. */
-	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_QUERYCTRL, arg);
 
 	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
@@ -903,6 +905,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -911,7 +914,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_G_CTRL, arg);
 }
 
@@ -994,6 +998,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -1008,7 +1013,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_S_CTRL, arg);
 }
 

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Tomasz Figa]

Hi Pavel,

Thanks for coming up with this proposal. Please see my comments below.

On Sun, Jun 3, 2018 at 6:01 AM Pavel Machek <pavel@ucw.cz> wrote:
>
> Hi!
>
> Ok, can I get any comments on this one?
> v4l2_open_complex("/file/with/descriptor", 0) can be used to open
> whole pipeline at once, and work if it as if it was one device.

I'm not convinced if we should really be piggy backing on libv4l, but
it's just a matter of where we put the code added by your patch, so
let's put that aside.

Who would be calling this function?

The scenario that I could think of is:
- legacy app would call open(/dev/video?), which would be handled by
libv4l open hook (v4l2_open()?),
- v4l2_open() would check if given /dev/video? figures in its list of
complex pipelines, for example by calling v4l2_open_complex() and
seeing if it succeeds,
- if it succeeds, the resulting fd would represent the complex
pipeline, otherwise it would just open the requested node directly.

I guess that could give some basic camera functionality on OMAP3-like hardware.

For most of the current generation of imaging subsystems (e.g. Intel
IPU3, Rockchip RKISP1) it's not enough. The reason is that there is
more to be handled by userspace than just setting controls:
 - configuring pixel formats, resolutions, crops, etc. through the
whole pipeline - I guess that could be preconfigured per use case
inside the configuration file, though,
 - forwarding buffers between capture and processing pipelines, i.e.
DQBUF raw frame from CSI2 video node and QBUF to ISP video node,
 - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
feedback loop - this might be optional if all we need is just ability
to capture some frames, but required for getting good quality,
 - actually mapping legacy controls into the above metadata,

I guess it's just a matter of adding further code to handle those,
though. However, it would build up a separate legacy framework that
locks us up into the legacy USB camera model, while we should rather
be leaning towards a more flexible framework, such as Android Camera
HALv3 or Pipewire. On top of such framework, we could just have a very
thin layer to emulate the legacy, single video node, camera.

Other than that, I agree that we should be able to have pre-generated
configuration files and use them to build and setup the pipeline.
That's actually what we have in camera HALs on Chrome OS (Android
camera HALv3 model adopted to Chrome OS).

Some minor comments for the code follow.

[snip]
> +int v4l2_get_fd_for_control(int fd, unsigned long control)
> +{
> +       int index = v4l2_get_index(fd);
> +       struct v4l2_controls_map *map;
> +       int lo = 0;
> +       int hi;
> +
> +       if (index < 0)
> +               return fd;
> +
> +       map = devices[index].map;
> +       if (!map)
> +               return fd;
> +       hi = map->num_controls;
> +
> +       while (lo < hi) {
> +               int i = (lo + hi) / 2;
> +               if (map->map[i].control == control) {
> +                       return map->map[i].fd;
> +               }
> +               if (map->map[i].control > control) {
> +                       hi = i;
> +                       continue;
> +               }
> +               if (map->map[i].control < control) {
> +                       lo = i+1;
> +                       continue;
> +               }
> +               printf("Bad: impossible condition in binary search\n");
> +               exit(1);
> +       }

Could we use bsearch() here?

> +       return fd;
> +}
> +
>  int v4l2_ioctl(int fd, unsigned long int request, ...)
>  {
>         void *arg;
>         va_list ap;
>         int result, index, saved_err;
> -       int is_capture_request = 0, stream_needs_locking = 0;
> +       int is_capture_request = 0, stream_needs_locking = 0,
> +           is_subdev_request = 0;
>
>         va_start(ap, request);
>         arg = va_arg(ap, void *);
> @@ -1076,18 +1115,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
>            ioctl, causing it to get sign extended, depending upon this behavior */
>         request = (unsigned int)request;
>
> +       /* FIXME */
>         if (devices[index].convert == NULL)
>                 goto no_capture_request;
>
>         /* Is this a capture request and do we need to take the stream lock? */
>         switch (request) {
> -       case VIDIOC_QUERYCAP:
>         case VIDIOC_QUERYCTRL:
>         case VIDIOC_G_CTRL:
>         case VIDIOC_S_CTRL:
>         case VIDIOC_G_EXT_CTRLS:
> -       case VIDIOC_TRY_EXT_CTRLS:
>         case VIDIOC_S_EXT_CTRLS:
> +               is_subdev_request = 1;
> +       case VIDIOC_QUERYCAP:
> +       case VIDIOC_TRY_EXT_CTRLS:

I'm not sure why we are moving those around. Is this perhaps related
to the FIXME comment above? If so, it would be helpful to have some
short explanation next to it.

>         case VIDIOC_ENUM_FRAMESIZES:
>         case VIDIOC_ENUM_FRAMEINTERVALS:
>                 is_capture_request = 1;
> @@ -1151,10 +1192,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
>         }
>
>         if (!is_capture_request) {
> +         int sub_fd;
>  no_capture_request:
> +                 sub_fd = fd;
> +               if (is_subdev_request) {
> +                 sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
> +               }

nit: Looks like something weird going on here with indentation.

>                 result = devices[index].dev_ops->ioctl(
>                                 devices[index].dev_ops_priv,
> -                               fd, request, arg);
> +                               sub_fd, request, arg);
>                 saved_err = errno;
>                 v4l2_log_ioctl(request, arg, result);
>                 errno = saved_err;
> @@ -1782,3 +1828,195 @@ int v4l2_get_control(int fd, int cid)
>                         (qctrl.maximum - qctrl.minimum) / 2) /
>                 (qctrl.maximum - qctrl.minimum);
>  }
> +
> +

nit: Double blank line.

> +int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
> +{
> +       int index;
> +       int i;
> +
> +       for (i=0; i<map->num_controls; i++) {
> +               if (map->map[i].fd <= 0) {
> +                       V4L2_LOG_ERR("v4l2_open_pipeline: Bad fd in map.\n");
> +                       return -1;
> +               }
> +               if (i>=1 && map->map[i].control <= map->map[i-1].control) {
> +                       V4L2_LOG_ERR("v4l2_open_pipeline: Controls not sorted.\n");
> +                       return -1;

I guess we could just sort them at startup with qsort()?

Best regards,
Tomasz

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3873 bytes --]

Hi!

> Thanks for coming up with this proposal. Please see my comments below.
> 
> > Ok, can I get any comments on this one?
> > v4l2_open_complex("/file/with/descriptor", 0) can be used to open
> > whole pipeline at once, and work if it as if it was one device.
> 
> I'm not convinced if we should really be piggy backing on libv4l, but
> it's just a matter of where we put the code added by your patch, so
> let's put that aside.

There was some talk about this before, and libv4l2 is what we came
with. Only libv4l2 is in position to propagate controls to right
devices.

> Who would be calling this function?
> 
> The scenario that I could think of is:
> - legacy app would call open(/dev/video?), which would be handled by
> libv4l open hook (v4l2_open()?),

I don't think that kind of legacy apps is in use any more. I'd prefer
not to deal with them.

> - v4l2_open() would check if given /dev/video? figures in its list of
> complex pipelines, for example by calling v4l2_open_complex() and
> seeing if it succeeds,

I'd rather not have v4l2_open_complex() called on devices. We could
test if argument is regular file and then call it... But again, that's
next step.

> - if it succeeds, the resulting fd would represent the complex
> pipeline, otherwise it would just open the requested node directly.

> I guess that could give some basic camera functionality on OMAP3-like hardware.

It definitely gives camera functionality on OMAP3. I'm using it to
take photos with Nokia N900.

> For most of the current generation of imaging subsystems (e.g. Intel
> IPU3, Rockchip RKISP1) it's not enough. The reason is that there is
> more to be handled by userspace than just setting controls:
>  - configuring pixel formats, resolutions, crops, etc. through the
> whole pipeline - I guess that could be preconfigured per use case
> inside the configuration file, though,

That may be future plan. Note that these can be preconfigured; unlike
controls propagation...

>  - forwarding buffers between capture and processing pipelines, i.e.
> DQBUF raw frame from CSI2 video node and QBUF to ISP video node,

My hardware does not need that, so I could not test it. I'll rely on
someone with required hardware to provide that.

(And you can take DQBUF and process it with software, at cost of
slightly higher CPU usage, right?)

>  - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
> feedback loop - this might be optional if all we need is just ability
> to capture some frames, but required for getting good quality,
>  - actually mapping legacy controls into the above metadata,

I'm not sure what 3A is. If you mean hardware histograms and friends,
yes, it would be nice to support that, but, again, statistics can be
computed in software.

> I guess it's just a matter of adding further code to handle those,
> though. However, it would build up a separate legacy framework that
> locks us up into the legacy USB camera model, while we should rather
> be leaning towards a more flexible framework, such as Android Camera
> HALv3 or Pipewire. On top of such framework, we could just have a very
> thin layer to emulate the legacy, single video node, camera.

Yes, we'll need something more advanced.

But.. we also need something to run the devices today, so that kernel
drivers can be tested and do not bitrot. That's why I'm doing this
work.

And I believe we should work in steps before getting there... controls
propagation can not be done from external application, so I'm starting
with it.

> Some minor comments for the code follow.

Ok, let me send this, then go through the comments.

Best regards,
							Pavel


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Tomasz Figa]

On Wed, Jun 6, 2018 at 5:46 PM Pavel Machek <pavel@ucw.cz> wrote:
>
> Hi!
>
> > Thanks for coming up with this proposal. Please see my comments below.
> >
> > > Ok, can I get any comments on this one?
> > > v4l2_open_complex("/file/with/descriptor", 0) can be used to open
> > > whole pipeline at once, and work if it as if it was one device.
> >
> > I'm not convinced if we should really be piggy backing on libv4l, but
> > it's just a matter of where we put the code added by your patch, so
> > let's put that aside.
>
> There was some talk about this before, and libv4l2 is what we came
> with. Only libv4l2 is in position to propagate controls to right
> devices.
>
> > Who would be calling this function?
> >
> > The scenario that I could think of is:
> > - legacy app would call open(/dev/video?), which would be handled by
> > libv4l open hook (v4l2_open()?),
>
> I don't think that kind of legacy apps is in use any more. I'd prefer
> not to deal with them.

In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
19"), Mauro has mentioned a number of those:

"open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
source ones (Skype, Chrome, ...)"

>
> > - v4l2_open() would check if given /dev/video? figures in its list of
> > complex pipelines, for example by calling v4l2_open_complex() and
> > seeing if it succeeds,
>
> I'd rather not have v4l2_open_complex() called on devices. We could
> test if argument is regular file and then call it... But again, that's
> next step.
>
> > - if it succeeds, the resulting fd would represent the complex
> > pipeline, otherwise it would just open the requested node directly.

What's the answer to my original question of who would be calling
v4l2_open_complex(), then?

>
> > I guess that could give some basic camera functionality on OMAP3-like hardware.
>
> It definitely gives camera functionality on OMAP3. I'm using it to
> take photos with Nokia N900.
>
> > For most of the current generation of imaging subsystems (e.g. Intel
> > IPU3, Rockchip RKISP1) it's not enough. The reason is that there is
> > more to be handled by userspace than just setting controls:
> >  - configuring pixel formats, resolutions, crops, etc. through the
> > whole pipeline - I guess that could be preconfigured per use case
> > inside the configuration file, though,
>
> That may be future plan. Note that these can be preconfigured; unlike
> controls propagation...
>
> >  - forwarding buffers between capture and processing pipelines, i.e.
> > DQBUF raw frame from CSI2 video node and QBUF to ISP video node,
>
> My hardware does not need that, so I could not test it. I'll rely on
> someone with required hardware to provide that.
>
> (And you can take DQBUF and process it with software, at cost of
> slightly higher CPU usage, right?)
>
> >  - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
> > feedback loop - this might be optional if all we need is just ability
> > to capture some frames, but required for getting good quality,
> >  - actually mapping legacy controls into the above metadata,
>
> I'm not sure what 3A is. If you mean hardware histograms and friends,
> yes, it would be nice to support that, but, again, statistics can be
> computed in software.

Auto-exposure, auto-white-balance, auto-focus. In complex camera
subsystems these need to be done in software. On most hardware
platforms, ISP provides necessary input data (statistics) and software
calculates required processing parameters.

>
> > I guess it's just a matter of adding further code to handle those,
> > though. However, it would build up a separate legacy framework that
> > locks us up into the legacy USB camera model, while we should rather
> > be leaning towards a more flexible framework, such as Android Camera
> > HALv3 or Pipewire. On top of such framework, we could just have a very
> > thin layer to emulate the legacy, single video node, camera.
>
> Yes, we'll need something more advanced.
>
> But.. we also need something to run the devices today, so that kernel
> drivers can be tested and do not bitrot. That's why I'm doing this
> work.

I guess the most important bit I missed then is what is the intended
use case for this. It seems to be related to my earlier, unanswered
question about who would be calliing v4l2_open_complex(), though.

What userspace applications would be used for this testing?

Best regards,
Tomasz

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3530 bytes --]

Hi!

> > > Who would be calling this function?
> > >
> > > The scenario that I could think of is:
> > > - legacy app would call open(/dev/video?), which would be handled by
> > > libv4l open hook (v4l2_open()?),
> >
> > I don't think that kind of legacy apps is in use any more. I'd prefer
> > not to deal with them.
> 
> In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> 19"), Mauro has mentioned a number of those:
> 
> "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> source ones (Skype, Chrome, ...)"

Yep, ok. Still would prefer not to deal with them.

(Opening additional fds behind application's back is quite nasty,
those apps should really switch to v4l2_ variants).

> > > - v4l2_open() would check if given /dev/video? figures in its list of
> > > complex pipelines, for example by calling v4l2_open_complex() and
> > > seeing if it succeeds,
> >
> > I'd rather not have v4l2_open_complex() called on devices. We could
> > test if argument is regular file and then call it... But again, that's
> > next step.
> >
> > > - if it succeeds, the resulting fd would represent the complex
> > > pipeline, otherwise it would just open the requested node directly.
> 
> What's the answer to my original question of who would be calling
> v4l2_open_complex(), then?

Application ready to deal with additional fds being
opened. contrib/test/sdlcam will be the first one.

We may do some magic to do v4l2_open_complex() in v4l2_open(), but I
believe that should be separate step.

> > >  - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
> > > feedback loop - this might be optional if all we need is just ability
> > > to capture some frames, but required for getting good quality,
> > >  - actually mapping legacy controls into the above metadata,
> >
> > I'm not sure what 3A is. If you mean hardware histograms and friends,
> > yes, it would be nice to support that, but, again, statistics can be
> > computed in software.
> 
> Auto-exposure, auto-white-balance, auto-focus. In complex camera
> subsystems these need to be done in software. On most hardware
> platforms, ISP provides necessary input data (statistics) and software
> calculates required processing parameters.

Ok, so... statistics support would be nice, but that is really
separate problem.

v4l2 already contains auto-exposure and auto-white-balance. I have
patches for auto-focus. But hardware statistics are not used.

> > Yes, we'll need something more advanced.
> >
> > But.. we also need something to run the devices today, so that kernel
> > drivers can be tested and do not bitrot. That's why I'm doing this
> > work.
> 
> I guess the most important bit I missed then is what is the intended
> use case for this. It seems to be related to my earlier, unanswered
> question about who would be calliing v4l2_open_complex(), though.
> 
> What userspace applications would be used for this testing?

Main use case is kernel testing.

Secondary use case is taking .jpg photos using sdlcam.

Test apps such as qv4l2 would be nice to have, and maybe I'll
experiment with capturing video somehow one day. I'm pretty sure it
will not be easy.

Oh and I guess a link to how well it works? See
https://www.youtube.com/watch?v=fH6zuK2OOVU .

Best regards,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Wed, 6 Jun 2018 12:01:50 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > > Who would be calling this function?
> > > >
> > > > The scenario that I could think of is:
> > > > - legacy app would call open(/dev/video?), which would be handled by
> > > > libv4l open hook (v4l2_open()?),  
> > >
> > > I don't think that kind of legacy apps is in use any more. I'd prefer
> > > not to deal with them.  
> > 
> > In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> > 19"), Mauro has mentioned a number of those:
> > 
> > "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> > source ones (Skype, Chrome, ...)"  
> 
> Yep, ok. Still would prefer not to deal with them.

I guess we'll end by needing to handle them. Anyway, now that PCs are
starting to come with complex cameras[1], we'll need to address this
with a focus on adding support for existing apps.

[1] we had a report from one specific model but I heard from a reliable
source that there are already other devices with similar issues.

> (Opening additional fds behind application's back is quite nasty,
> those apps should really switch to v4l2_ variants).

Only closed source apps use the LD_PRELOAD hack. All the others
use v4l2_ variants, but, as Nicolas mentioned at the other
thread, there are a number of problems with the current approach.

Perhaps is time for us to not be limited to the current ABI, writing
a new API from scratch, and then adding a compatibility layer to be
used by apps that rely on v4l2_ variants, in order to avoid breaking
the ABI and keep providing LD_PRELOAD. We can then convert the apps
we use/care most to use the new ABI.

> 
> > > > - v4l2_open() would check if given /dev/video? figures in its list of
> > > > complex pipelines, for example by calling v4l2_open_complex() and
> > > > seeing if it succeeds,  
> > >
> > > I'd rather not have v4l2_open_complex() called on devices. We could
> > > test if argument is regular file and then call it... But again, that's
> > > next step.
> > >  
> > > > - if it succeeds, the resulting fd would represent the complex
> > > > pipeline, otherwise it would just open the requested node directly.  
> > 
> > What's the answer to my original question of who would be calling
> > v4l2_open_complex(), then?  
> 
> Application ready to deal with additional fds being
> opened. contrib/test/sdlcam will be the first one.
> 
> We may do some magic to do v4l2_open_complex() in v4l2_open(), but I
> believe that should be separate step.

In order to avoid breaking the ABI for existing apps, v4l2_open() should
internally call v4l2_open_complex() (or whatever we call it at the new
API design).

> > > >  - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
> > > > feedback loop - this might be optional if all we need is just ability
> > > > to capture some frames, but required for getting good quality,
> > > >  - actually mapping legacy controls into the above metadata,  
> > >
> > > I'm not sure what 3A is. If you mean hardware histograms and friends,
> > > yes, it would be nice to support that, but, again, statistics can be
> > > computed in software.  
> > 
> > Auto-exposure, auto-white-balance, auto-focus. In complex camera
> > subsystems these need to be done in software. On most hardware
> > platforms, ISP provides necessary input data (statistics) and software
> > calculates required processing parameters.  
> 
> Ok, so... statistics support would be nice, but that is really
> separate problem.
> 
> v4l2 already contains auto-exposure and auto-white-balance. I have
> patches for auto-focus. But hardware statistics are not used.

Feel free to submit the auto-focus patches anytime. With all 3A
algos there (even on a non-optimal way using hw stats), it will
make easier for us when designing a solution that would work for
both IMAP3 and ISP (and likely be generic enough for other hardware).

For the full complex hardware solution, though, it is probably better
to fork v4l-utils into a separate project, in order to do the development
without affecting current users of it, merging it back only after we'll
be sure that existing apps will keep working with v4l2_foo() functions
and LD_PRELOAD.

Anyway, this is something that it makes sense to discuss during the
Complex Camera Workshop.

> > > Yes, we'll need something more advanced.
> > >
> > > But.. we also need something to run the devices today, so that kernel
> > > drivers can be tested and do not bitrot. That's why I'm doing this
> > > work.  
> > 
> > I guess the most important bit I missed then is what is the intended
> > use case for this. It seems to be related to my earlier, unanswered
> > question about who would be calliing v4l2_open_complex(), though.
> > 
> > What userspace applications would be used for this testing?  
> 
> Main use case is kernel testing.
> 
> Secondary use case is taking .jpg photos using sdlcam.

If a v4l2_complex_open() will, for now, be something that we don't
export publicly, using it only for the tools already at libv4l2,
I don't see much troubles on adding it, but I would hate to have to
stick with this ABI. Otherwise, we should analyze it after having
a bigger picture. So, better to wait for the Complex Camera
Workshop before adding this.

Btw, would you be able to join us there (either locally or
remotely via Google Hangouts)?
 
> Test apps such as qv4l2 would be nice to have, and maybe I'll
> experiment with capturing video somehow one day. I'm pretty sure it
> will not be easy.

Capture video is a must have for PCs. The final solution should
take it into account.

> 
> Oh and I guess a link to how well it works? See
> https://www.youtube.com/watch?v=fH6zuK2OOVU .
> 
> Best regards,
> 								Pavel

Thanks,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 4169 bytes --]

Hi!

> > > > I don't think that kind of legacy apps is in use any more. I'd prefer
> > > > not to deal with them.  
> > > 
> > > In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> > > 19"), Mauro has mentioned a number of those:
> > > 
> > > "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> > > source ones (Skype, Chrome, ...)"  
> > 
> > Yep, ok. Still would prefer not to deal with them.
> 
> I guess we'll end by needing to handle them. Anyway, now that PCs are
> starting to come with complex cameras[1], we'll need to address this
> with a focus on adding support for existing apps.
> 
> [1] we had a report from one specific model but I heard from a reliable
> source that there are already other devices with similar issues.

Well, now that the issue hit PCs, we'll get help from Linux distributors.

> > (Opening additional fds behind application's back is quite nasty,
> > those apps should really switch to v4l2_ variants).
> 
> Only closed source apps use the LD_PRELOAD hack. All the others
> use v4l2_ variants, but, as Nicolas mentioned at the other
> thread, there are a number of problems with the current approach.
> 
> Perhaps is time for us to not be limited to the current ABI, writing
> a new API from scratch, and then adding a compatibility layer to be
> used by apps that rely on v4l2_ variants, in order to avoid breaking
> the ABI and keep providing LD_PRELOAD. We can then convert the apps
> we use/care most to use the new ABI.

I believe we can support current features using existing libv4l2 abi
-- complex cameras should work as well as dumb ones do.

...something like that is certainly needed for testing.

But yes, long-term better interface is needed -- and code should not
be in library but in separate process.

> > Application ready to deal with additional fds being
> > opened. contrib/test/sdlcam will be the first one.
> > 
> > We may do some magic to do v4l2_open_complex() in v4l2_open(), but I
> > believe that should be separate step.
> 
> In order to avoid breaking the ABI for existing apps, v4l2_open() should
> internally call v4l2_open_complex() (or whatever we call it at the new
> API design).

Ok... everyone wants that. I can do that.

> > Ok, so... statistics support would be nice, but that is really
> > separate problem.
> > 
> > v4l2 already contains auto-exposure and auto-white-balance. I have
> > patches for auto-focus. But hardware statistics are not used.
> 
> Feel free to submit the auto-focus patches anytime. With all 3A
> algos there (even on a non-optimal way using hw stats), it will
> make easier for us when designing a solution that would work for
> both IMAP3 and ISP (and likely be generic enough for other hardware).

Let me finish the control propagation, first. My test hardware
supporting autofocus is N900, so I need control propagation to be able
to test autofocus.

> > Secondary use case is taking .jpg photos using sdlcam.
> 
> If a v4l2_complex_open() will, for now, be something that we don't
> export publicly, using it only for the tools already at libv4l2,
> I don't see much troubles on adding it, but I would hate to have to
> stick with this ABI. Otherwise, we should analyze it after having
> a bigger picture. So, better to wait for the Complex Camera
> Workshop before adding this.
> 
> Btw, would you be able to join us there (either locally or
> remotely via Google Hangouts)?

I don't plan going to Japan. Hangouts... I'm not big fan of Google,
but I'll try. I'm in GMT+2.

> > Test apps such as qv4l2 would be nice to have, and maybe I'll
> > experiment with capturing video somehow one day. I'm pretty sure it
> > will not be easy.
> 
> Capture video is a must have for PCs. The final solution should
> take it into account.

N900 is quite slow for JPEG compression (and I don't really want
solution depending on hardware extras like DSP)... so that will be
fun.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[PATCH, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 12697 bytes --]

Hi!

> > We may do some magic to do v4l2_open_complex() in v4l2_open(), but I
> > believe that should be separate step.
> 
> In order to avoid breaking the ABI for existing apps, v4l2_open() should
> internally call v4l2_open_complex() (or whatever we call it at the new
> API design).

Ok. Here's updated patch. open_complex() now takes fd. Any other
issues?

Best regards,
								Pavel

diff --git a/lib/include/libv4l2.h b/lib/include/libv4l2.h
index ea1870d..a0ec0a9 100644
--- a/lib/include/libv4l2.h
+++ b/lib/include/libv4l2.h
@@ -58,6 +58,10 @@ LIBV4L_PUBLIC extern FILE *v4l2_log_file;
    invalid memory address will not lead to failure with errno being EFAULT,
    as it would with a real ioctl, but will cause libv4l2 to break, and you
    get to keep both pieces.
+
+   You can open complex pipelines by passing ".cv" file with pipeline
+   description to v4l2_open(). libv4l2 will open all the required
+   devices automatically in that case.
 */
 
 LIBV4L_PUBLIC int v4l2_open(const char *file, int oflag, ...);
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 1924c91..1ee697a 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -104,6 +104,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+	struct v4l2_controls_map *map;
 };
 
 /* From v4l2-plugin.c */
@@ -130,4 +131,20 @@ static inline void v4l2_plugin_cleanup(void *plugin_lib, void *plugin_priv,
 extern const char *v4l2_ioctls[];
 void v4l2_log_ioctl(unsigned long int request, void *arg, int result);
 
+
+struct v4l2_control_map {
+	unsigned long control;
+	int fd;
+};
+
+struct v4l2_controls_map {
+	int main_fd;
+	int num_fds;
+	int num_controls;
+	struct v4l2_control_map map[];
+};
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags);
+LIBV4L_PUBLIC int v4l2_get_fd_for_control(int fd, unsigned long control);
+
 #endif
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 2db25d1..ac430f0 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -70,6 +70,8 @@
 #include <sys/types.h>
 #include <sys/mman.h>
 #include <sys/stat.h>
+#include <dirent.h>
+
 #include "libv4l2.h"
 #include "libv4l2-priv.h"
 #include "libv4l-plugin.h"
@@ -618,6 +620,8 @@ static void v4l2_update_fps(int index, struct v4l2_streamparm *parm)
 		devices[index].fps = 0;
 }
 
+static int v4l2_open_complex(int fd, int v4l2_flags);
+
 int v4l2_open(const char *file, int oflag, ...)
 {
 	int fd;
@@ -641,6 +645,21 @@ int v4l2_open(const char *file, int oflag, ...)
 	if (fd == -1)
 		return fd;
 
+	int len = strlen(file);
+	char *end = ".cv";
+	int len2 = strlen(end);
+	if ((len > len2) && (!strcmp(file + len - len2, end))) {
+		/* .cv extension */
+		struct stat sb;
+
+		if (fstat(fd, &sb) == 0) {
+			if ((sb.st_mode & S_IFMT) == S_IFREG) {
+				return v4l2_open_complex(fd, 0);
+			}
+		}
+		
+	}
+
 	if (v4l2_fd_open(fd, 0) == -1) {
 		int saved_err = errno;
 
@@ -787,6 +806,8 @@ no_capture:
 	if (index >= devices_used)
 		devices_used = index + 1;
 
+	devices[index].map = NULL;
+
 	/* Note we always tell v4lconvert to optimize src fmt selection for
 	   our default fps, the only exception is the app explicitly selecting
 	   a frame rate using the S_PARM ioctl after a S_FMT */
@@ -1056,12 +1077,47 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+int v4l2_get_fd_for_control(int fd, unsigned long control)
+{
+	int index = v4l2_get_index(fd);
+	struct v4l2_controls_map *map;
+	int lo = 0;
+	int hi;
+
+	if (index < 0)
+		return fd;
+
+	map = devices[index].map;
+	if (!map)
+		return fd;
+	hi = map->num_controls;
+
+	while (lo < hi) {
+		int i = (lo + hi) / 2;
+		if (map->map[i].control == control) {
+			return map->map[i].fd;
+		}
+		if (map->map[i].control > control) {
+			hi = i;
+			continue;
+		}
+		if (map->map[i].control < control) {
+			lo = i+1;
+			continue;
+		}
+		printf("Bad: impossible condition in binary search\n");
+		exit(1);
+	}
+	return fd;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
 	va_list ap;
 	int result, index, saved_err;
-	int is_capture_request = 0, stream_needs_locking = 0;
+	int is_capture_request = 0, stream_needs_locking = 0, 
+	    is_subdev_request = 0;
 
 	va_start(ap, request);
 	arg = va_arg(ap, void *);
@@ -1076,18 +1132,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	   ioctl, causing it to get sign extended, depending upon this behavior */
 	request = (unsigned int)request;
 
+	/* FIXME */
 	if (devices[index].convert == NULL)
 		goto no_capture_request;
 
 	/* Is this a capture request and do we need to take the stream lock? */
 	switch (request) {
-	case VIDIOC_QUERYCAP:
 	case VIDIOC_QUERYCTRL:
 	case VIDIOC_G_CTRL:
 	case VIDIOC_S_CTRL:
 	case VIDIOC_G_EXT_CTRLS:
-	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_S_EXT_CTRLS:
+		is_subdev_request = 1;
+	case VIDIOC_QUERYCAP:
+	case VIDIOC_TRY_EXT_CTRLS:
 	case VIDIOC_ENUM_FRAMESIZES:
 	case VIDIOC_ENUM_FRAMEINTERVALS:
 		is_capture_request = 1;
@@ -1151,10 +1209,15 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
 	}
 
 	if (!is_capture_request) {
+		int sub_fd;
 no_capture_request:
+		sub_fd = fd;
+		if (is_subdev_request) {
+			sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
+		}
 		result = devices[index].dev_ops->ioctl(
 				devices[index].dev_ops_priv,
-				fd, request, arg);
+				sub_fd, request, arg);
 		saved_err = errno;
 		v4l2_log_ioctl(request, arg, result);
 		errno = saved_err;
@@ -1782,3 +1845,194 @@ int v4l2_get_control(int fd, int cid)
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }
+
+int v4l2_open_pipeline(struct v4l2_controls_map *map, int v4l2_flags)
+{
+	int index;
+	int i;
+
+	for (i=0; i<map->num_controls; i++) {
+		if (map->map[i].fd <= 0) {
+			V4L2_LOG_ERR("v4l2_open_pipeline: Bad fd in map.\n");
+			return -1;
+		}
+		if (i>=1 && map->map[i].control <= map->map[i-1].control) {
+			V4L2_LOG_ERR("v4l2_open_pipeline: Controls not sorted.\n");
+			return -1;
+		}
+	}
+
+	i = v4l2_fd_open(map->main_fd, v4l2_flags);
+	index = v4l2_get_index(map->main_fd);
+	devices[index].map = map;
+	return i;
+}
+
+static void scan_devices(char **device_names, int *device_fds, int num)
+{
+	struct dirent **namelist;
+	int n;
+	char *class_v4l = "/sys/class/video4linux";
+
+	n = scandir(class_v4l, &namelist, NULL, alphasort);
+	if (n < 0) {
+		perror("scandir");
+		return;
+	}
+	
+	while (n--) {
+		if (namelist[n]->d_name[0] != '.') {
+			char filename[1024], content[1024];
+			sprintf(filename, "%s/%s/name", class_v4l, namelist[n]->d_name);
+			FILE *f = fopen(filename, "r");
+			if (!f) {
+				printf("Strange, can't open %s", filename);
+			} else {
+				fgets(content, 1024, f);
+				fclose(f);
+
+				int i;
+				for (i = num-1; i >=0; i--) {
+					if (!strcmp(content, device_names[i])) {
+						sprintf(filename, "/dev/%s", namelist[n]->d_name);
+						device_fds[i] = open(filename, O_RDWR);
+						if (device_fds[i] < 0) {
+							V4L2_LOG_ERR("Error opening %s: %m\n", filename);
+						}
+					}
+				}
+			}
+		}
+		free(namelist[n]);
+	}
+	free(namelist);
+  
+}
+
+static int v4l2_open_complex(int fd, int v4l2_flags)
+{
+#define perr(s) V4L2_LOG_ERR("open_complex: " s "\n")
+#define BUF 256
+	FILE *f = fdopen(fd, "r");
+
+	int res = -1;
+	char buf[BUF];
+	int version, num_modes, num_devices, num_controls;
+	int dev, control;
+
+	if (!f) {
+		perr("open of .cv file failed: %m");
+		goto err;
+	}
+
+	if (fscanf(f, "Complex Video: %d\n", &version) != 1) {
+		perr(".cv file does not have required header");
+		goto close;
+	}
+
+	if (version != 0) {
+		perr(".cv file has unknown version");
+		goto close;
+	}
+  
+	if (fscanf(f, "#modes: %d\n", &num_modes) != 1) {
+		perr("could not parse modes");
+		goto close;
+	}
+
+	if (num_modes != 1) {
+		perr("only single mode is supported for now");
+		goto close;
+	}
+
+	if (fscanf(f, "Mode: %s\n", buf) != 1) {
+		perr("could not parse mode name");
+		goto close;
+	}
+
+	if (fscanf(f, " #devices: %d\n", &num_devices) != 1) {
+		perr("could not parse number of devices");
+		goto close;
+	}
+#define MAX_DEVICES 16
+	char *device_names[MAX_DEVICES] = { NULL, };
+	int device_fds[MAX_DEVICES];
+	if (num_devices > MAX_DEVICES) {
+		perr("too many devices");
+		goto close;
+	}
+  
+	for (dev = 0; dev < num_devices; dev++) {
+		int tmp;
+		if (fscanf(f, "%d: ", &tmp) != 1) {
+			perr("could not parse device");
+			goto free_devices;
+		}
+		if (tmp != dev) {
+			perr("bad device number");
+			goto free_devices;
+		}
+		fgets(buf, BUF, f);
+		device_names[dev] = strdup(buf);
+		device_fds[dev] = -1;
+	}
+
+	scan_devices(device_names, device_fds, num_devices);
+
+	for (dev = 0; dev < num_devices; dev++) {
+		if (device_fds[dev] == -1) {
+			perr("Could not open all required devices");
+			goto close_devices;
+		}
+	}
+
+	if (fscanf(f, " #controls: %d\n", &num_controls) != 1) {
+		perr("can not parse number of controls");
+		goto close_devices;
+	}
+
+	struct v4l2_controls_map *map = malloc(sizeof(struct v4l2_controls_map) +
+					       num_controls*sizeof(struct v4l2_control_map));
+
+	map->num_controls = num_controls;
+	map->num_fds = num_devices;
+	map->main_fd = device_fds[0];
+  
+	for (control = 0; control < num_controls; control++) {
+		unsigned long num;
+		int dev;
+		if (fscanf(f, "0x%lx: %d\n", &num, &dev) != 2) {
+			perr("could not parse control");
+			goto free_map;
+		}
+		if ((dev < 0) || (dev >= num_devices)) {
+			perr("device out of range");
+			goto free_map;
+		}
+		map->map[control].control = num;
+		map->map[control].fd = device_fds[dev];
+	}
+	if (fscanf(f, "%s", buf) > 0) {
+		perr("junk at end of file");
+		goto free_map;
+	}
+
+	res = v4l2_open_pipeline(map, v4l2_flags);
+
+	if (res < 0) {
+free_map:
+		free(map);
+close_devices:
+		for (dev = 0; dev < num_devices; dev++)
+			close(device_fds[dev]);
+	}
+free_devices:
+	for (dev = 0; dev < num_devices; dev++) {
+		free(device_names[dev]);
+	}
+close:
+	fclose(f);
+err:
+	return res;
+}
+
diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
index 59f28b1..c1e6f93 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.c
+++ b/lib/libv4lconvert/control/libv4lcontrol.c
@@ -863,6 +863,7 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 	struct v4l2_queryctrl *ctrl = arg;
 	int retval;
 	uint32_t orig_id = ctrl->id;
+	int fd;
 
 	/* if we have an exact match return it */
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
@@ -872,8 +873,9 @@ int v4lcontrol_vidioc_queryctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
 	/* find out what the kernel driver would respond. */
-	retval = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	retval = data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_QUERYCTRL, arg);
 
 	if ((data->priv_flags & V4LCONTROL_SUPPORTS_NEXT_CTRL) &&
@@ -903,6 +905,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -911,7 +914,8 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_G_CTRL, arg);
 }
 
@@ -994,6 +998,7 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int fd;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -1008,7 +1013,8 @@ int v4lcontrol_vidioc_s_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	fd = v4l2_get_fd_for_control(data->fd, ctrl->id);
+	return data->dev_ops->ioctl(data->dev_ops_priv, fd,
 			VIDIOC_S_CTRL, arg);
 }
 


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

HI!

> > > Thanks for coming up with this proposal. Please see my comments below.
> > >
> > > > Ok, can I get any comments on this one?
> > > > v4l2_open_complex("/file/with/descriptor", 0) can be used to open
> > > > whole pipeline at once, and work if it as if it was one device.
> > >
> > > I'm not convinced if we should really be piggy backing on libv4l, but
> > > it's just a matter of where we put the code added by your patch, so
> > > let's put that aside.
> >
> > There was some talk about this before, and libv4l2 is what we came
> > with. Only libv4l2 is in position to propagate controls to right
> > devices.
> >
> > > Who would be calling this function?
> > >
> > > The scenario that I could think of is:
> > > - legacy app would call open(/dev/video?), which would be handled by
> > > libv4l open hook (v4l2_open()?),
> >
> > I don't think that kind of legacy apps is in use any more. I'd prefer
> > not to deal with them.
> 
> In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> 19"), Mauro has mentioned a number of those:
> 
> "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> source ones (Skype, Chrome, ...)"

Thanks for thread pointer... I may be able to get in using hangouts.

Anyway, there's big difference between open("/dev/video0") and
v4l2_open("/dev/video0"). I don't care about the first one, but yes we
should be able to support the second one eventually.

And I don't think Mauro says apps like Camorama are of open() kind.

Best regards,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Tomasz Figa]

On Wed, Jun 6, 2018 at 7:51 PM Pavel Machek <pavel@ucw.cz> wrote:
>
> HI!
>
> > > > Thanks for coming up with this proposal. Please see my comments below.
> > > >
> > > > > Ok, can I get any comments on this one?
> > > > > v4l2_open_complex("/file/with/descriptor", 0) can be used to open
> > > > > whole pipeline at once, and work if it as if it was one device.
> > > >
> > > > I'm not convinced if we should really be piggy backing on libv4l, but
> > > > it's just a matter of where we put the code added by your patch, so
> > > > let's put that aside.
> > >
> > > There was some talk about this before, and libv4l2 is what we came
> > > with. Only libv4l2 is in position to propagate controls to right
> > > devices.
> > >
> > > > Who would be calling this function?
> > > >
> > > > The scenario that I could think of is:
> > > > - legacy app would call open(/dev/video?), which would be handled by
> > > > libv4l open hook (v4l2_open()?),
> > >
> > > I don't think that kind of legacy apps is in use any more. I'd prefer
> > > not to deal with them.
> >
> > In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> > 19"), Mauro has mentioned a number of those:
> >
> > "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> > source ones (Skype, Chrome, ...)"
>
> Thanks for thread pointer... I may be able to get in using hangouts.
>
> Anyway, there's big difference between open("/dev/video0") and
> v4l2_open("/dev/video0"). I don't care about the first one, but yes we
> should be able to support the second one eventually.
>
> And I don't think Mauro says apps like Camorama are of open() kind.

I don't think there is much difference between open() and v4l2_open(),
since the former can be changed to the latter using LD_PRELOAD.

If we simply add v4l2_open_complex() to libv4l, we would have to get
developers of the applications (regardless of whether they use open()
or v4l2_open()) to also support v4l2_open_complex(). For testing
purposes of kernel developers it would work indeed, but I wonder if it
goes anywhere beyond that.

If all we need is some code to be able to test kernel camera drivers,
I don't think there is any big problem in adding v4l2_open_complex()
to libv4l. However, we must either ensure that either:
a) it's not going to be widely used
OR
b) it is designed well enough to cover the complex cases I mentioned
and which are likely to represent most of the hardware in the wild.

Best regards,
Tomasz

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1781 bytes --]

Hi!

> > Thanks for thread pointer... I may be able to get in using hangouts.
> >
> > Anyway, there's big difference between open("/dev/video0") and
> > v4l2_open("/dev/video0"). I don't care about the first one, but yes we
> > should be able to support the second one eventually.
> >
> > And I don't think Mauro says apps like Camorama are of open() kind.
> 
> I don't think there is much difference between open() and v4l2_open(),
> since the former can be changed to the latter using LD_PRELOAD.

Well, if everyone thinks opening more than one fd in v4l2_open() is
okay, I can do that. Probably "if argument is regular file and has .mc
extension, use open_complex"?  

> If we simply add v4l2_open_complex() to libv4l, we would have to get
> developers of the applications (regardless of whether they use open()
> or v4l2_open()) to also support v4l2_open_complex(). For testing
> purposes of kernel developers it would work indeed, but I wonder if it
> goes anywhere beyond that.

I'd like people to think "is opening multiple fds okay at this
moment?" before switching to v4l2_open_complex(). But I guess I'm too careful.

> If all we need is some code to be able to test kernel camera drivers,
> I don't think there is any big problem in adding v4l2_open_complex()
> to libv4l. However, we must either ensure that either:
> a) it's not going to be widely used
> OR
> b) it is designed well enough to cover the complex cases I mentioned
> and which are likely to represent most of the hardware in the wild.

.mc descriptors should be indeed extensible enough for that.

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Mauro Carvalho Chehab]

Em Wed, 6 Jun 2018 12:51:16 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> > > > The scenario that I could think of is:
> > > > - legacy app would call open(/dev/video?), which would be handled by
> > > > libv4l open hook (v4l2_open()?),  
> > >
> > > I don't think that kind of legacy apps is in use any more. I'd prefer
> > > not to deal with them.  
> > 
> > In another thread ("[ANN v2] Complex Camera Workshop - Tokyo - Jun,
> > 19"), Mauro has mentioned a number of those:
> > 
> > "open source ones (Camorama, Cheese, Xawtv, Firefox, Chromium, ...) and closed
> > source ones (Skype, Chrome, ...)"  
> 
> Thanks for thread pointer... I may be able to get in using hangouts.
> 
> Anyway, there's big difference between open("/dev/video0") and
> v4l2_open("/dev/video0"). I don't care about the first one, but yes we
> should be able to support the second one eventually.
> 
> And I don't think Mauro says apps like Camorama are of open() kind.

All open source apps we care use v4l2_open() & friends. the ones
that use just open() work via LD_PRELOAD. It is a hack, but it
was needed when libv4l was added (as there were lots of apps
to be touched). Also, we had problems on that time with closed
source app developers. I guess nowadays, among v4l-specific
apps, only closed source ones use just open().

Haven't check how browsers open cameras, though. A quick look at the
Fedora 60 dependencies, though, doesn't show libv4l:

	https://rpmfind.net/linux/RPM/fedora/devel/rawhide/x86_64/f/firefox-60.0.1-5.fc29.x86_64.html

It might be statically linking libv4l, or maybe they rely on something
else (like java/flash/...), but I guess it is more likely that they're
just using open() somehow. The same kind of issue may also be present
on other browsers and on java libraries.

Thanks,
Mauro

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3186 bytes --]

Hi!

> > +       while (lo < hi) {
> > +               int i = (lo + hi) / 2;
> > +               if (map->map[i].control == control) {
> > +                       return map->map[i].fd;
> > +               }
> > +               if (map->map[i].control > control) {
> > +                       hi = i;
> > +                       continue;
> > +               }
> > +               if (map->map[i].control < control) {
> > +                       lo = i+1;
> > +                       continue;
> > +               }
> > +               printf("Bad: impossible condition in binary search\n");
> > +               exit(1);
> > +       }
> 
> Could we use bsearch() here?

We could, but it will mean passing function pointers etc. It would
make sense if we want to do sorting.

> > @@ -1076,18 +1115,20 @@ int v4l2_ioctl(int fd, unsigned long int request, ...)
> >            ioctl, causing it to get sign extended, depending upon this behavior */
> >         request = (unsigned int)request;
> >
> > +       /* FIXME */
> >         if (devices[index].convert == NULL)
> >                 goto no_capture_request;
> >
> >         /* Is this a capture request and do we need to take the stream lock? */
> >         switch (request) {
> > -       case VIDIOC_QUERYCAP:
> >         case VIDIOC_QUERYCTRL:
> >         case VIDIOC_G_CTRL:
> >         case VIDIOC_S_CTRL:
> >         case VIDIOC_G_EXT_CTRLS:
> > -       case VIDIOC_TRY_EXT_CTRLS:
> >         case VIDIOC_S_EXT_CTRLS:
> > +               is_subdev_request = 1;
> > +       case VIDIOC_QUERYCAP:
> > +       case VIDIOC_TRY_EXT_CTRLS:
> 
> I'm not sure why we are moving those around. Is this perhaps related
> to the FIXME comment above? If so, it would be helpful to have some
> short explanation next to it.

I want to do controls propagation only on ioctls that manipulate
controls, so I need to group them together. The FIXME comment is not
related.

> >
> >         if (!is_capture_request) {
> > +         int sub_fd;
> >  no_capture_request:
> > +                 sub_fd = fd;
> > +               if (is_subdev_request) {
> > +                 sub_fd = v4l2_get_fd_for_control(index, ((struct v4l2_queryctrl *) arg)->id);
> > +               }
> 
> nit: Looks like something weird going on here with indentation.

Fixed.

> > @@ -1782,3 +1828,195 @@ int v4l2_get_control(int fd, int cid)
> >                         (qctrl.maximum - qctrl.minimum) / 2) /
> >                 (qctrl.maximum - qctrl.minimum);
> >  }
> > +
> > +
> 
> nit: Double blank line.

Fixed.

> > +               if (i>=1 && map->map[i].control <= map->map[i-1].control) {
> > +                       V4L2_LOG_ERR("v4l2_open_pipeline: Controls not sorted.\n");
> > +                       return -1;
> 
> I guess we could just sort them at startup with qsort()?

We could... but I'd prefer them sorted on-disk, as it is written very
seldom, but needs to be readed on every device open.

Thanks for review,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [RFC, libv4l]: Make libv4l2 usable on devices with complex pipeline

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1715 bytes --]

Hi!

> I guess that could give some basic camera functionality on OMAP3-like hardware.

Yeah, and that is the goal.

> For most of the current generation of imaging subsystems (e.g. Intel
> IPU3, Rockchip RKISP1) it's not enough. The reason is that there is
> more to be handled by userspace than just setting controls:
>  - configuring pixel formats, resolutions, crops, etc. through the
> whole pipeline - I guess that could be preconfigured per use case
> inside the configuration file, though,
>  - forwarding buffers between capture and processing pipelines, i.e.
> DQBUF raw frame from CSI2 video node and QBUF to ISP video node,
>  - handling metadata CAPTURE and OUTPUT buffers controlling the 3A
> feedback loop - this might be optional if all we need is just ability
> to capture some frames, but required for getting good quality,
>  - actually mapping legacy controls into the above metadata,

I just wanted to add few things:

It seems IPU3 and RKISP1 is really similar to what I have on
N900. Forwarding frames between parts of processing pipeline is not
neccessary, but the other parts are there.

There are also two points where you can gather the image data, either
(almost) raw GRBG10 data from the sensor, or scaled YUV data ready for
display. [And how to display that data without CPU involvement is
another, rather big, topic.]

Anyway, legacy applications expect simple webcams with bad pictures,
low resolution, and no AF support. And we should be able to provide
them with just that.

Best regards,

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: fix integer overflow

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 599 bytes --]

Hi!

> > This bit me while trying to use absolute exposure time on Nokia N900:
> > 
> > Can someone apply it to libv4l2 tree? Could I get some feedback on the
> > other patches? Is this the way to submit patches to libv4l2?
> 
> Yes, it is. But I do need a Signed-off-by from you.

Ok, that should be it for today.

I also put all but the first patch into the git, at

https://gitlab.com/tui/v4l-utils/tree/merge

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: fix integer overflow

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 509 bytes --]

Hi!

> > This bit me while trying to use absolute exposure time on Nokia N900:
> > 
> > Can someone apply it to libv4l2 tree? Could I get some feedback on the
> > other patches? Is this the way to submit patches to libv4l2?
> 
> Yes, it is. But I do need a Signed-off-by from you.

Ping? Can I get you to apply the patches?

Thanks,
								Pavel
								
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 36649 bytes --]

Add simple SDL-based application for capturing photos. Manual
focus/gain/exposure can be set, flash can be controlled and
autofocus/autogain can be selected if camera supports that.

It is already useful for testing autofocus/autogain improvements to
the libraries on Nokia N900.

Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/configure.ac b/configure.ac
index f30d66d..2c8ad7e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -88,6 +88,247 @@ LIBDVBV5_DOMAIN="libdvbv5"
 AC_DEFINE([LIBDVBV5_DOMAIN], "libdvbv5", [libdvbv5 domain])
 AC_SUBST(LIBDVBV5_DOMAIN)
 
+# Configure paths for SDL
+# Sam Lantinga 9/21/99
+# stolen from Manish Singh
+# stolen back from Frank Belew
+# stolen from Manish Singh
+# Shamelessly stolen from Owen Taylor
+#
+# Changelog:
+# * also look for SDL2.framework under Mac OS X
+
+# serial 1
+
+dnl AM_PATH_SDL2([MINIMUM-VERSION, [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND]]])
+dnl Test for SDL, and define SDL_CFLAGS and SDL_LIBS
+dnl
+AC_DEFUN([AM_PATH_SDL2],
+[dnl 
+dnl Get the cflags and libraries from the sdl2-config script
+dnl
+AC_ARG_WITH(sdl-prefix,[  --with-sdl-prefix=PFX   Prefix where SDL is installed (optional)],
+            sdl_prefix="$withval", sdl_prefix="")
+AC_ARG_WITH(sdl-exec-prefix,[  --with-sdl-exec-prefix=PFX Exec prefix where SDL is installed (optional)],
+            sdl_exec_prefix="$withval", sdl_exec_prefix="")
+AC_ARG_ENABLE(sdltest, [  --disable-sdltest       Do not try to compile and run a test SDL program],
+		    , enable_sdltest=yes)
+AC_ARG_ENABLE(sdlframework, [  --disable-sdlframework Do not search for SDL2.framework],
+        , search_sdl_framework=yes)
+
+AC_ARG_VAR(SDL2_FRAMEWORK, [Path to SDL2.framework])
+
+  min_sdl_version=ifelse([$1], ,2.0.0,$1)
+
+  if test "x$sdl_prefix$sdl_exec_prefix" = x ; then
+    PKG_CHECK_MODULES([SDL], [sdl2 >= $min_sdl_version],
+           [sdl_pc=yes],
+           [sdl_pc=no])
+  else
+    sdl_pc=no
+    if test x$sdl_exec_prefix != x ; then
+      sdl_config_args="$sdl_config_args --exec-prefix=$sdl_exec_prefix"
+      if test x${SDL2_CONFIG+set} != xset ; then
+        SDL2_CONFIG=$sdl_exec_prefix/bin/sdl2-config
+      fi
+    fi
+    if test x$sdl_prefix != x ; then
+      sdl_config_args="$sdl_config_args --prefix=$sdl_prefix"
+      if test x${SDL2_CONFIG+set} != xset ; then
+        SDL2_CONFIG=$sdl_prefix/bin/sdl2-config
+      fi
+    fi
+  fi
+
+  if test "x$sdl_pc" = xyes ; then
+    no_sdl=""
+    SDL2_CONFIG="pkg-config sdl2"
+  else
+    as_save_PATH="$PATH"
+    if test "x$prefix" != xNONE && test "$cross_compiling" != yes; then
+      PATH="$prefix/bin:$prefix/usr/bin:$PATH"
+    fi
+    AC_PATH_PROG(SDL2_CONFIG, sdl2-config, no, [$PATH])
+    PATH="$as_save_PATH"
+    no_sdl=""
+
+    if test "$SDL2_CONFIG" = "no" -a "x$search_sdl_framework" = "xyes"; then
+      AC_MSG_CHECKING(for SDL2.framework)
+      if test "x$SDL2_FRAMEWORK" != x; then
+        sdl_framework=$SDL2_FRAMEWORK
+      else
+        for d in / ~/ /System/; do
+          if test -d "$dLibrary/Frameworks/SDL2.framework"; then
+            sdl_framework="$dLibrary/Frameworks/SDL2.framework"
+          fi
+        done
+      fi
+
+      if test -d $sdl_framework; then
+        AC_MSG_RESULT($sdl_framework)
+        sdl_framework_dir=`dirname $sdl_framework`
+        SDL_CFLAGS="-F$sdl_framework_dir -Wl,-framework,SDL2 -I$sdl_framework/include"
+        SDL_LIBS="-F$sdl_framework_dir -Wl,-framework,SDL2"
+      else
+        no_sdl=yes
+      fi
+    fi
+
+    if test "$SDL2_CONFIG" != "no"; then
+      if test "x$sdl_pc" = "xno"; then
+        AC_MSG_CHECKING(for SDL - version >= $min_sdl_version)
+        SDL_CFLAGS=`$SDL2_CONFIG $sdl_config_args --cflags`
+        SDL_LIBS=`$SDL2_CONFIG $sdl_config_args --libs`
+      fi
+
+      sdl_major_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\1/'`
+      sdl_minor_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\2/'`
+      sdl_micro_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\3/'`
+      if test "x$enable_sdltest" = "xyes" ; then
+        ac_save_CFLAGS="$CFLAGS"
+        ac_save_CXXFLAGS="$CXXFLAGS"
+        ac_save_LIBS="$LIBS"
+        CFLAGS="$CFLAGS $SDL_CFLAGS"
+        CXXFLAGS="$CXXFLAGS $SDL_CFLAGS"
+        LIBS="$LIBS $SDL_LIBS"
+dnl
+dnl Now check if the installed SDL is sufficiently new. (Also sanity
+dnl checks the results of sdl2-config to some extent
+dnl
+      rm -f conf.sdltest
+      AC_TRY_RUN([
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "SDL.h"
+
+char*
+my_strdup (char *str)
+{
+  char *new_str;
+  
+  if (str)
+    {
+      new_str = (char *)malloc ((strlen (str) + 1) * sizeof(char));
+      strcpy (new_str, str);
+    }
+  else
+    new_str = NULL;
+  
+  return new_str;
+}
+
+int main (int argc, char *argv[])
+{
+  int major, minor, micro;
+  char *tmp_version;
+
+  /* This hangs on some systems (?)
+  system ("touch conf.sdltest");
+  */
+  { FILE *fp = fopen("conf.sdltest", "a"); if ( fp ) fclose(fp); }
+
+  /* HP/UX 9 (%@#!) writes to sscanf strings */
+  tmp_version = my_strdup("$min_sdl_version");
+  if (sscanf(tmp_version, "%d.%d.%d", &major, &minor, &micro) != 3) {
+     printf("%s, bad version string\n", "$min_sdl_version");
+     exit(1);
+   }
+
+   if (($sdl_major_version > major) ||
+      (($sdl_major_version == major) && ($sdl_minor_version > minor)) ||
+      (($sdl_major_version == major) && ($sdl_minor_version == minor) && ($sdl_micro_version >= micro)))
+    {
+      return 0;
+    }
+  else
+    {
+      printf("\n*** 'sdl2-config --version' returned %d.%d.%d, but the minimum version\n", $sdl_major_version, $sdl_minor_version, $sdl_micro_version);
+      printf("*** of SDL required is %d.%d.%d. If sdl2-config is correct, then it is\n", major, minor, micro);
+      printf("*** best to upgrade to the required version.\n");
+      printf("*** If sdl2-config was wrong, set the environment variable SDL2_CONFIG\n");
+      printf("*** to point to the correct copy of sdl2-config, and remove the file\n");
+      printf("*** config.cache before re-running configure\n");
+      return 1;
+    }
+}
+
+],, no_sdl=yes,[echo $ac_n "cross compiling; assumed OK... $ac_c"])
+        CFLAGS="$ac_save_CFLAGS"
+        CXXFLAGS="$ac_save_CXXFLAGS"
+        LIBS="$ac_save_LIBS"
+
+      fi
+      if test "x$sdl_pc" = "xno"; then
+        if test "x$no_sdl" = "xyes"; then
+          AC_MSG_RESULT(no)
+        else
+          AC_MSG_RESULT(yes)
+        fi
+      fi
+    fi
+  fi
+  if test "x$no_sdl" = x ; then
+     ifelse([$2], , :, [$2])
+  else
+     if test "$SDL2_CONFIG" = "no" ; then
+       echo "*** The sdl2-config script installed by SDL could not be found"
+       echo "*** If SDL was installed in PREFIX, make sure PREFIX/bin is in"
+       echo "*** your path, or set the SDL2_CONFIG environment variable to the"
+       echo "*** full path to sdl2-config."
+     else
+       if test -f conf.sdltest ; then
+        :
+       else
+          echo "*** Could not run SDL test program, checking why..."
+          CFLAGS="$CFLAGS $SDL_CFLAGS"
+          CXXFLAGS="$CXXFLAGS $SDL_CFLAGS"
+          LIBS="$LIBS $SDL_LIBS"
+          AC_TRY_LINK([
+#include <stdio.h>
+#include "SDL.h"
+
+int main(int argc, char *argv[])
+{ return 0; }
+#undef  main
+#define main K_and_R_C_main
+],      [ return 0; ],
+        [ echo "*** The test program compiled, but did not run. This usually means"
+          echo "*** that the run-time linker is not finding SDL or finding the wrong"
+          echo "*** version of SDL. If it is not finding SDL, you'll need to set your"
+          echo "*** LD_LIBRARY_PATH environment variable, or edit /etc/ld.so.conf to point"
+          echo "*** to the installed location  Also, make sure you have run ldconfig if that"
+          echo "*** is required on your system"
+	  echo "***"
+          echo "*** If you have an old version installed, it is best to remove it, although"
+          echo "*** you may also be able to get things to work by modifying LD_LIBRARY_PATH"],
+        [ echo "*** The test program failed to compile or link. See the file config.log for the"
+          echo "*** exact error that occured. This usually means SDL was incorrectly installed"
+          echo "*** or that you have moved SDL since it was installed. In the latter case, you"
+          echo "*** may want to edit the sdl2-config script: $SDL2_CONFIG" ])
+          CFLAGS="$ac_save_CFLAGS"
+          CXXFLAGS="$ac_save_CXXFLAGS"
+          LIBS="$ac_save_LIBS"
+       fi
+     fi
+     SDL_CFLAGS=""
+     SDL_LIBS=""
+     ifelse([$3], , :, [$3])
+  fi
+  AC_SUBST(SDL_CFLAGS)
+  AC_SUBST(SDL_LIBS)
+  rm -f conf.sdltest
+])
+
+dnl Check for SDL
+SDL_VERSION=2.0
+AM_PATH_SDL2($SDL_VERSION, sdl_pkgconfig=yes, sdl_pkgconfig=no)
+
+AM_CONDITIONAL([HAVE_SDL], [test x$sdl_pkgconfig = xyes])
+
 # Define localedir
 AC_DEFUN([V4L_EXPAND_PREFIX], [
 	$1=$2
@@ -432,5 +673,6 @@ compile time options summary
     libudev		: $have_libudev
     QT version		: $QT_VERSION
     ALSA support	: $alsa_pkgconfig
+    SDL support		: $sdl_pkgconfig
 
 EOF
diff --git a/contrib/test/Makefile.am b/contrib/test/Makefile.am
index 4641e21..dd06cc1 100644
--- a/contrib/test/Makefile.am
+++ b/contrib/test/Makefile.am
@@ -16,6 +16,10 @@ if HAVE_GLU
 noinst_PROGRAMS += v4l2gl
 endif
 
+if HAVE_SDL
+noinst_PROGRAMS += sdlcam
+endif
+
 driver_test_SOURCES = driver-test.c
 driver_test_LDADD = ../../utils/libv4l2util/libv4l2util.la
 
@@ -31,6 +35,10 @@ v4l2gl_SOURCES = v4l2gl.c
 v4l2gl_LDFLAGS = $(X11_LIBS) $(GL_LIBS) $(GLU_LIBS) $(ARGP_LIBS)
 v4l2gl_LDADD = ../../lib/libv4l2/libv4l2.la ../../lib/libv4lconvert/libv4lconvert.la
 
+sdlcam_LDFLAGS = $(JPEG_LIBS) $(SDL_LIBS) ../../lib/libv4l2/.libs/libv4l2.a  ../../lib/libv4lconvert/.libs/libv4lconvert.a
+sdlcam_CFLAGS = -I../..
+v4l2gl_LDADD = 
+
 mc_nextgen_test_CFLAGS = $(LIBUDEV_CFLAGS)
 mc_nextgen_test_LDFLAGS = $(LIBUDEV_LIBS)
 
diff --git a/contrib/test/sdlcam.c b/contrib/test/sdlcam.c
new file mode 100644
index 0000000..16d1bef
--- /dev/null
+++ b/contrib/test/sdlcam.c
@@ -0,0 +1,1093 @@
+/*
+   Digital still camera.
+
+   SDL based, suitable for camera phone such as Nokia N900. In
+   particular, we support focus, gain and exposure control, but not
+   aperture control or lens zoom.
+
+   Copyright 2017 Pavel Machek, LGPLv2 or later.
+*/
+
+#include <time.h>
+#include <errno.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/ioctl.h>
+#include <fcntl.h>
+
+#include <jpeglib.h>
+
+#include "libv4l2.h"
+#include <linux/videodev2.h>
+#include "libv4l-plugin.h"
+
+#include <SDL2/SDL.h>
+#include <SDL2/SDL_image.h>
+
+static double dtime(void)
+{
+	static double start = 0.0;
+	struct timeval now;
+
+	gettimeofday(&now, NULL);
+
+	double n = now.tv_sec + now.tv_usec / 1000000.;
+	if (!start)
+		start = n;
+	return n - start;
+}
+
+static long v4l2_g_ctrl(int fd, long id)
+{
+	int res;
+	struct v4l2_control ctrl;
+	ctrl.id = id;
+	res = v4l2_ioctl(fd, VIDIOC_G_CTRL, &ctrl);
+	if (res < 0)
+		printf("Get control %ld failed\n", id);
+	return ctrl.value;
+}
+
+static int v4l2_s_ctrl(int fd, long id, long value)
+{
+	int res;
+	struct v4l2_control ctrl;
+	ctrl.id = id;
+	ctrl.value = value;
+	res = v4l2_ioctl(fd, VIDIOC_S_CTRL, &ctrl);
+	if (res < 0)
+		printf("Set control %lx %ld failed\n", id, value);
+	return res;
+}
+
+
+static int v4l2_set_focus(int fd, int diopt)
+{
+	if (v4l2_s_ctrl(fd, V4L2_CID_FOCUS_ABSOLUTE, diopt) < 0) {
+		printf("Could not set focus\n");
+	}
+	return 0;
+}
+
+#define SIZE 1296*984*3
+struct dev_info {
+	int fd;
+	struct v4l2_format fmt;
+
+	unsigned char buf[SIZE];
+	int debug;
+#define D_TIMING 1
+};
+
+struct sdl {
+	SDL_Window *window;
+	SDL_Surface *screen, *liveview;
+
+	int wx, wy; /* Window size */
+	int sx, sy; /* Live view size */
+	int bx, by; /* Border size */
+	int nx, ny; /* Number of buttons */
+	int factor;
+
+	/* These should go separately */
+	int do_focus, do_exposure, do_flash, do_white;
+	double focus_min;
+	
+	int fd;
+
+	struct dev_info *dev;
+};
+
+typedef struct {
+	uint8_t r, g, b, alpha;
+} pixel;
+
+#define d_raw 1
+
+static void sfc_put_pixel(SDL_Surface* liveview, int x, int y, pixel *p)
+{
+	Uint32* p_liveview = (Uint32*)liveview->pixels;
+	p_liveview += y*liveview->w+x;
+	*p_liveview = SDL_MapRGBA(liveview->format, p->r, p->g, p->b, p->alpha);
+}
+
+static void sdl_begin_paint(struct sdl *m)
+{
+	/* Fill the surface white */
+	SDL_FillRect(m->liveview, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+
+	SDL_LockSurface(m->liveview);
+}
+
+static void sdl_finish_paint(struct sdl *m) {
+	SDL_UnlockSurface(m->liveview);
+	SDL_Rect rcDest = { m->bx, m->by, m->sx, m->sy };
+
+	SDL_BlitSurface(m->liveview, NULL, m->screen, &rcDest);
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+  
+static void sdl_paint_image(struct sdl *m, char **xpm, int x, int y) {
+	SDL_Surface *image = IMG_ReadXPMFromArray(xpm);
+	if (!image) {
+		printf("IMG_Load: %s\n", IMG_GetError());
+		exit(1);
+	}
+
+	int x_pos = x - image->w/2, y_pos = y - image->h/2;
+
+	SDL_Rect rcDest = { x_pos, y_pos, image->w, image->h };
+	int r = SDL_BlitSurface ( image, NULL, m->screen, &rcDest );
+
+	if (r) {
+		printf("Error blitting: %s\n", SDL_GetError());
+		exit(1);
+	}
+	SDL_FreeSurface ( image );
+}
+
+static void cam_exposure_limits(struct sdl *m, struct v4l2_queryctrl *qctrl)
+{
+	qctrl->id = V4L2_CID_EXPOSURE_ABSOLUTE;
+	
+	if (v4l2_ioctl(m->fd, VIDIOC_QUERYCTRL, qctrl)) {
+		printf("Exposure absolute limits failed\n");
+		exit(1);
+	}
+
+	if (qctrl->minimum < 500)
+		qctrl->minimum = 500;
+}
+
+static void cam_set_exposure(struct sdl *m, double v)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	double res;
+	double range;
+	struct v4l2_queryctrl qctrl = { .id = cid };
+	struct v4l2_control ctrl = { .id = cid };
+
+	cam_exposure_limits(m, &qctrl);
+
+	if (v4l2_ioctl(m->fd, VIDIOC_G_CTRL, &ctrl)) {
+		printf("Can't get exposure parameters\n");
+		exit(1);
+	}
+
+	range = log2(qctrl.maximum) - log2(qctrl.minimum);
+	res = log2(qctrl.minimum) + v*range;
+	res = exp2(res);
+
+	v4l2_s_ctrl(m->fd, V4L2_CID_EXPOSURE_ABSOLUTE, res);
+}
+
+static double cam_convert_exposure(struct sdl *m, int v)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	double res;
+	struct v4l2_queryctrl qctrl = { .id = cid };
+
+	cam_exposure_limits(m, &qctrl);
+	res = (log2(v) - log2(qctrl.minimum)) / (log2(qctrl.maximum) - log2(qctrl.minimum));
+
+	return res;
+}
+
+static double cam_get_exposure(struct sdl *m)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	struct v4l2_control ctrl = { .id = cid };
+
+	if (v4l2_ioctl(m->fd, VIDIOC_G_CTRL, &ctrl))
+		return -1;
+
+	return cam_convert_exposure(m, ctrl.value);
+}
+
+static void cam_set_focus(struct sdl *m, double val)
+{
+	v4l2_s_ctrl(m->fd, V4L2_CID_FOCUS_ABSOLUTE, (val * m->focus_min) * (1023. / 20));
+}
+
+static double cam_convert_focus(struct sdl *m, double diopter)
+{
+	return diopter / m->focus_min;
+}
+
+static double cam_get_focus_diopter(struct sdl *m)
+{
+	return (v4l2_g_ctrl(m->fd, V4L2_CID_FOCUS_ABSOLUTE) * 20.) / 1023.;
+}
+
+static double cam_get_focus(struct sdl *m)
+{
+	return cam_convert_focus(m, cam_get_focus_diopter(m));
+}
+
+static void sdl_line_color(struct sdl *m, int x1, int y1, int x2, int y2, Uint32 color)
+{
+	SDL_Rect rcDest = { x1, y1, x2-x1+1, y2-y1+1};
+
+	SDL_FillRect(m->screen, &rcDest, color);
+}
+
+static void sdl_line(struct sdl *m, int x1, int y1, int x2, int y2)
+{
+	sdl_line_color(m, x1, y1, x2, y2, SDL_MapRGB( m->liveview->format, 255, 255, 255 ));
+}
+
+static void sdl_digit(struct sdl *m, int x, int y, int s, int i)
+{
+	unsigned char gr[] = { 0x5f, 0x0a, 0x76, 0x7a, 0x2b,
+			       0x79, 0x7d, 0x1a, 0x7f, 0x7b };
+	unsigned char g = gr[i];
+	/*
+              10
+	    01  02
+              20
+            04  08
+	      40
+	*/
+
+	if (g & 1) sdl_line(m, x, y, x, y+s);
+	if (g & 2) sdl_line(m, x+s, y, x+s, y+s);
+	if (g & 4) sdl_line(m, x, y+s, x, y+s+s);
+	if (g & 8) sdl_line(m, x+s, y+s, x+s, y+s+s);
+
+	if (g & 0x10) sdl_line(m, x, y, x+s, y);
+	if (g & 0x20) sdl_line(m, x, y+s, x+s, y+s);
+	if (g & 0x40) sdl_line(m, x, y+s+s, x+s, y+s+s);
+}
+
+static void sdl_number(struct sdl *m, int x, int y, int s, int digits, int i)
+{
+	int tot = s * 5;
+	x += tot * digits;
+	for (int j=0; j<digits; j++) {
+		sdl_digit(m, x+2, y+4, s*3, i%10);
+		i /= 10;
+		x -= tot;
+	}
+}
+
+static void sdl_paint_ui_iso(struct sdl *m, double y_, int i)
+{
+	static char *iso_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		".x..xx..x.......",
+		".x.x...x.x......",
+		".x..x..x.x......",
+		".x...x.x.x......",
+		".x.xx...x.......",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+
+	int x = m->bx - 10;
+	int y = m->by+m->sy*y_;
+
+	sdl_number(m, x-35, y-10, 1, 3, i);
+	sdl_paint_image(m, iso_xpm, x, y);
+}
+
+static void sdl_paint_ui_exposure(struct sdl *m, int t)
+{
+	static char *time_1_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"......x.........",
+		".....x..........",
+		"....x...........",
+		"...x............",
+		"................",
+		".xxx.xxxx.xxxx..",
+		"x....x....x.....",
+		".xx..xxx..x.....",
+		"...x.x....x.....",
+		"...x.x....x.....",
+		"xxx..xxxx.xxxx..",
+		"................",
+	};
+	int x = m->wx-m->bx + 30;
+	int y = m->by+m->sy*cam_convert_exposure(m, 1000000/t);
+
+	sdl_number(m, x-35, y-10, 1, 3, t);
+	sdl_paint_image(m, time_1_xpm, x, y);
+}
+
+static void sdl_paint_boolean(struct sdl *m, char **image, int x, int y, int yes)
+{
+	static char *not_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"......xxxxx.....",
+		"....xx.....xx...",
+		"...x.........x..",
+		"..x........xx.x.",
+		"..x......xx...x.",
+		".x.....xx......x",
+		".x...xx........x",
+		"..xxx.........x.",
+		"..x...........x.",
+		"...x.........x..",
+		"....xx.....xx...",
+		"......xxxxx.....",
+	};
+
+	sdl_paint_image(m, image, x, y);
+	if (!yes)
+		sdl_paint_image(m, not_xpm,  16+x, y);      
+}
+
+static void sdl_paint_button(struct sdl *m, char **image, int x, int y, int yes)
+{
+	int bsx = m->wx/m->nx;
+	int bsy = m->wy/m->ny;
+	if (x < 0)
+		x += m->nx;
+	if (y < 0)
+		y += m->ny;
+	x = bsx/2 + x*bsx;
+	y = bsy/2 + y*bsy;
+	sdl_paint_boolean(m, image, x, y, yes);
+}
+
+static void sdl_paint_ui_focus(struct sdl *m, int f)
+{
+	static char *cm_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"....###..#.#.##.",
+		"...#.....##.#..#",
+		"...#.....#..#..#",
+		"...#.....#..#..#",
+		"....###..#..#..#",
+		"................",
+	};
+	double dioptr = 1/(f/100.);
+	int x = m->bx+cam_convert_focus(m, dioptr)*m->sx;
+	int y = m->by - 20;
+
+	if (dioptr > m->focus_min)
+		return;
+	sdl_paint_image(m, cm_xpm, x, y);
+	sdl_number(m, x-12, y-15, 1, 3, f);
+}
+
+static void sdl_paint_ui(struct sdl *m)
+{
+	static char *wait_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"....########....",
+		".....#....#.....",
+		".....#....#.....",
+		"......#..#......",
+		".......##.......",
+		"......#..#......",
+		".....#....#.....",
+		".....#....#.....",
+		"....########....",
+	};
+
+	static char *ok_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"...............#",
+		"............###.",
+		"..........##....",
+		"#.......##......",
+		".#.....#........",
+		"..#...#.........",
+		"..#..#..........",
+		"...##...........",
+		"...#............",
+	};
+
+	static char *exit_xpm[] = {
+		"16 9 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"....x......x....",
+		".....x....x.....",
+		"......x..x......",
+		".......xx.......",
+		".......xx.......",
+		"......x..x......",
+		".....x....x.....",
+		"....x......x....",
+		"................",
+	};
+
+	static char *af_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..x..........",
+		"................",
+		"................",
+	};
+
+	static char *ae_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..xxxxxxx....",
+		"................",
+		"................",
+	};
+    
+	static char *focus_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"###..........###",
+		"#..............#",
+		"#.....####.....#",
+		".....#....#.....",
+		".....#....#.....",
+		"#.....####.....#",
+		"#..............#",
+		"###..........###",
+		"................",
+		"................",
+	};
+
+	static char *flash_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",		
+		"..........#.....",
+		"........##......",
+		".......##.......",
+		"......##........",
+		".....########...",
+		"..........##....",
+		".......#.##.....",
+		".......###......",
+		".......####.....",
+		"................",
+		"................",
+	};
+
+	static char *wb_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"#.....#..####...",
+		"#.....#..#...#..",
+		"#..#..#..####...",
+		"#..#..#..#...#..",
+		".##.##...####...",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+/* Template for more xpm's:
+	static char *empty_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+*/
+	SDL_FillRect(m->screen, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+
+	{
+		/* Paint grid */
+		int x, y;
+		int nx = m->nx;
+		for (x=1; x<nx; x++) {
+			int x_ = (x*m->wx)/nx;
+			sdl_line_color(m, x_, 1, x_, m->wy-1, SDL_MapRGB( m->liveview->format, 40, 40, 40 ));
+		}
+
+		int ny = m->ny;
+		for (y=1; y<nx; y++) {
+			int y_ = (y*m->wy)/ny;
+			sdl_line_color(m, 1, y_, m->wx-1, y_, SDL_MapRGB( m->liveview->format, 40, 40, 40 ));
+		}
+				       
+	}
+
+	sdl_paint_image(m, wait_xpm,  m->wx/2,     m->wy/2);
+
+	sdl_paint_ui_focus(m, 100);
+	sdl_paint_ui_focus(m, 40);
+	sdl_paint_ui_focus(m, 25);
+	sdl_paint_ui_focus(m, 16);	
+	sdl_paint_ui_focus(m, 10);
+	sdl_paint_ui_focus(m, 8);
+	sdl_paint_ui_focus(m, 6);
+
+	sdl_paint_button(m, af_xpm, 0,  0, m->do_focus);
+	sdl_paint_button(m, ae_xpm, -1, 0, m->do_exposure);
+
+	sdl_paint_button(m, exit_xpm,   0, -1, 1);
+	sdl_paint_button(m, flash_xpm,  1, -1, m->do_flash);
+	sdl_paint_button(m, wb_xpm,     2, -1, m->do_white);
+	sdl_paint_button(m, focus_xpm, -2, -1, 1);
+	sdl_paint_button(m, ok_xpm,    -1, -1, 1);
+
+	sdl_paint_ui_exposure(m, 10);
+	sdl_paint_ui_exposure(m, 100);
+	sdl_paint_ui_exposure(m, 999);
+
+	sdl_paint_ui_iso(m, 0/4., 100);
+	sdl_paint_ui_iso(m, 1/4., 200);
+	sdl_paint_ui_iso(m, 2/4., 400);
+	sdl_paint_ui_iso(m, 3/4., 800);
+
+	SDL_UpdateWindowSurface(m->window);
+}
+
+static double usec_to_time(double v)
+{
+	return 1/(v*.000001);
+}
+
+static void sdl_status(struct sdl *m, double avg)
+{
+	int ox = m->bx;
+	int oy = m->by+m->sy;
+	SDL_Rect rcDest = { ox, oy, m->sx, 25 /* m->by */ };
+
+	SDL_FillRect(m->screen, &rcDest, SDL_MapRGB( m->liveview->format, 30, 30, 30 ));
+	ox+=40;
+	sdl_number(m, ox, oy, 2, 3, avg*1000);
+
+	{
+		double focus, gain, exposure;
+
+		exposure = v4l2_g_ctrl(m->fd, V4L2_CID_EXPOSURE_ABSOLUTE);
+		gain = v4l2_g_ctrl(m->fd, 0x00980913);
+		focus = cam_get_focus_diopter(m);
+
+		ox+=40;
+		double x = usec_to_time(exposure);
+		if (x > 999) x = 999;
+		sdl_number(m, ox, oy, 2, 3, x);
+
+		ox+=40;
+		x = (gain / 10);
+		sdl_number(m, ox, oy, 2, 1, x);
+
+		ox+=20;
+		x = focus; /* diopters */
+		if (x == 0)
+			x = 999;
+		else
+			x = 100/x; /* centimeters */
+		sdl_number(m, ox, oy, 2, 3, x);
+	}
+
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+
+static void fmt_print(struct v4l2_format *fmt)
+{
+	int f;
+	printf("Format: %dx%d. ", fmt->fmt.pix.width, fmt->fmt.pix.height);
+	printf("%x ", fmt->fmt.pix.pixelformat);
+	f = fmt->fmt.pix.pixelformat;
+	for (int i=0; i<4; i++) {
+		printf("%c", f & 0xff);
+		f >>= 8;
+	}
+	printf("\n");
+}
+
+static pixel buf_pixel(struct v4l2_format *fmt, unsigned char *buf, int x, int y)
+{
+	pixel p = { 0, 0, 0, 0 };
+	int pos = x + y*fmt->fmt.pix.width;
+
+	p.alpha = 128;
+
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+		{
+			short *b2 = (void *)buf;
+			x &= ~1;
+			y &= ~1;
+			p.g = b2[x + y*fmt->fmt.pix.width] /4;
+			p.r = b2[x + y*fmt->fmt.pix.width+1] /4;
+			p.b = b2[x + (y+1)*fmt->fmt.pix.width] /4;
+		}
+		break;
+
+	case V4L2_PIX_FMT_RGB24:
+		pos *= 3;
+		p.r = buf[pos];
+		p.g = buf[pos+1];
+		p.b = buf[pos+2];
+		break;
+
+	default:
+		printf("Wrong pixel format!\n");
+		fmt_print(fmt);
+		exit(1);
+	}
+
+	return p;
+}
+
+static char *fmt_name(struct v4l2_format *fmt)
+{
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+		return "GRBG10";
+	case V4L2_PIX_FMT_RGB24:
+		return "RGB24";
+	default:
+		return "unknown";
+	}
+}
+
+static void sdl_handle_focus(struct sdl *m, float how)
+{
+	v4l2_set_control(m->fd, V4L2_CID_FOCUS_ABSOLUTE, 65535. * how);
+}
+
+static void sdl_key(struct sdl *m, int c)
+{
+	switch (c) {
+	case 27: exit(1); break;
+	case 'q': sdl_handle_focus(m, 0.); break;
+	case 'w': sdl_handle_focus(m, 1/6.); break;
+	case 'e': sdl_handle_focus(m, 1/3.); break;
+	case 'r': sdl_handle_focus(m, 1/2.); break;
+	case 't': sdl_handle_focus(m, 1/1); break;
+	case 'y': sdl_handle_focus(m, 1/.8); break;
+	case 'u': sdl_handle_focus(m, 1/.5); break;
+	case 'i': sdl_handle_focus(m, 1/.2); break;
+	case 'o': sdl_handle_focus(m, 1/.1); break;
+	case 'p': sdl_handle_focus(m, 1/.05); break;
+	case SDLK_SPACE: /* save_image(); */ printf("Should save jpeg.\n"); break;
+	default: printf("Unknown key %d / %c", c, c);
+	}
+}
+
+static int render_statistics(struct sdl *m)
+{
+	pixel white;
+	double focus, gain, exposure;
+
+	white.r = (Uint8)0xff;
+	white.g = (Uint8)0xff;
+	white.b = (Uint8)0xff;
+	white.alpha = (Uint8)128;
+
+	exposure = cam_get_exposure(m);
+	gain = v4l2_get_control(m->fd, 0x00980913) / 65535.;
+	focus = cam_get_focus(m);
+
+	for (int x=0; x<m->sx && x<m->sx*focus; x++)
+		sfc_put_pixel(m->liveview, x, 0, &white);
+
+	for (int y=0; y<m->sy && y<m->sy*gain; y++)
+		sfc_put_pixel(m->liveview, 0, y, &white);
+
+	for (int y=0; y<m->sy && y<m->sy*exposure; y++)
+		sfc_put_pixel(m->liveview, m->sx-1, y, &white);
+
+	for (int x=0; x<m->sx; x++)
+		sfc_put_pixel(m->liveview, x, m->sy-1, &white);
+
+	return 0;
+}
+
+static void sdl_render(struct sdl *m, unsigned char *buf, struct v4l2_format *fmt)
+{
+	if (!m->window) 
+		return;
+	sdl_begin_paint(m);    
+
+	for (int y = 0; y < m->sy; y++)
+		for (int x = 0; x < m->sx; x++) {
+			pixel p = buf_pixel(fmt, buf, x*m->factor, y*m->factor);
+			p.alpha = 128;
+			sfc_put_pixel(m->liveview, x, y, &p);
+		}
+
+	render_statistics(m);
+	sdl_finish_paint(m);
+}
+
+static void sdl_sync_settings(struct sdl *m)
+{
+	printf("Autofocus: "); v4l2_s_ctrl(m->fd, V4L2_CID_FOCUS_AUTO, m->do_focus);
+	printf("Autogain: " ); v4l2_s_ctrl(m->fd, V4L2_CID_AUTOGAIN, m->do_exposure);
+	printf("Autowhite: "); v4l2_s_ctrl(m->fd, V4L2_CID_AUTO_WHITE_BALANCE, m->do_white);
+	v4l2_s_ctrl(m->fd, 0x009c0901, m->do_flash ? 2 : 0);
+}
+
+static void sdl_init(struct sdl *m, struct dev_info *dev, int _factor)
+{
+	m->fd = dev->fd;
+	m->dev = dev;
+
+	if (SDL_Init(SDL_INIT_VIDEO) < 0) {
+		printf("Could not init SDL\n");
+		exit(1);
+	}
+
+	atexit(SDL_Quit);
+
+	m->wx = 800;
+	m->wy = 429;
+
+	m->window = SDL_CreateWindow("Camera", SDL_WINDOWPOS_UNDEFINED,
+				     SDL_WINDOWPOS_UNDEFINED, m->wx, m->wy,
+				     SDL_WINDOW_SHOWN);
+	if (m->window == NULL) {
+		printf("Window could not be created! SDL_Error: %s\n", SDL_GetError());
+		exit(1);
+	}
+
+	m->screen = SDL_GetWindowSurface(m->window);
+	if (!m->screen) {
+		printf("Couldn't create screen\n");
+		exit(1);
+	}
+
+	m->sx = dev->fmt.fmt.pix.width;
+	m->sy = dev->fmt.fmt.pix.height;
+	m->factor = _factor;
+
+	m->sx /= m->factor;
+	m->sy /= m->factor;
+
+	m->bx = (m->wx-m->sx)/2;
+	m->by = (m->wy-m->sy)/2;
+
+	m->nx = 6;
+	m->ny = 4;
+
+	m->liveview = SDL_CreateRGBSurface(0,m->sx,m->sy,32,0,0,0,0);
+	if (!m->liveview) {
+		printf("Couldn't create liveview\n");
+		exit(1);
+	}
+
+	m->do_flash = 1;
+	m->do_focus = 0;
+	m->do_exposure = 1;
+	m->focus_min = 5;
+	sdl_paint_ui(m);
+	sdl_sync_settings(m);
+}
+
+static struct sdl sdl;
+
+static void pgm_write(struct dev_info *dev, struct v4l2_format *fmt, unsigned char *img, char *out_name)
+{
+	FILE *fout;
+	int size = fmt->fmt.pix.width * fmt->fmt.pix.height;
+	char swapped[size*2];
+
+	fout = fopen(out_name, "w");
+	if (!fout) {
+		perror("Cannot open image");
+		exit(EXIT_FAILURE);
+	}
+	switch (fmt->fmt.pix.pixelformat) {
+		/* ?? 	cinfo.in_color_space = JCS_YCbCr; */
+	case V4L2_PIX_FMT_SGRBG10:
+		printf("ok\n");
+		break;
+	default:
+		printf("Bad pixel format\n");
+		exit(1);
+	}
+
+	for (int i=0; i<size*2; i+=2) {
+		swapped[i] = img[i+1];
+		swapped[i+1] = img[i];
+	}
+
+	fprintf(fout, "P5\n# SDLcam raw image\n# ");
+	{
+		double focus, gain, exposure;
+
+		exposure = v4l2_get_control(dev->fd, V4L2_CID_EXPOSURE_ABSOLUTE) / 65536.;
+		gain = v4l2_get_control(dev->fd, 0x00980913) / 65536.;
+		focus = v4l2_get_control(dev->fd, V4L2_CID_FOCUS_ABSOLUTE) / 65536.;
+
+		fprintf(fout, "Exposure %f, gain %f, focus %f\n", exposure, gain, focus);
+	}
+	fprintf(fout, "%d %d\n1023\n",
+		fmt->fmt.pix.width, fmt->fmt.pix.height);
+	fwrite(swapped, size, 2, fout);
+	fclose(fout);
+}
+
+static void any_write(struct dev_info *dev)
+{
+	char name[1024];
+	unsigned char *buf;
+	time_t t = time(NULL);
+
+	buf = dev->buf;
+
+	if (1) {
+		sprintf(name, "/data/tmp/delme_%d.%s", (int) t, "pgm");
+
+		if (dev->fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_SGRBG10)
+			printf("Not in bayer10, can't write raw.\n");
+		else
+			pgm_write(dev, &dev->fmt, buf, name);
+	}
+}
+
+static void sdl_mouse(struct sdl *m, SDL_Event event)
+{
+	int ax = 0, ay = 0;
+	int nx = event.button.x / (m->wx/m->nx), ny = event.button.y / (m->wy/m->ny);
+	if (nx > m->nx/2)
+		nx -= m->nx;
+	if (ny > m->ny/2)
+		ny -= m->ny;
+
+	printf("Button %d %d\n", nx, ny);
+	switch (ny) {
+	case 0:
+		switch (nx) {
+		case 0:
+			m->do_focus ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case -1:
+			m->do_exposure ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		}
+		break;
+	case -1:
+		switch (nx) {
+		case 0:
+			exit(0);
+		case 1:
+			m->do_flash ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case 2:
+			m->do_white ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case -2:
+			v4l2_s_ctrl(m->fd, V4L2_CID_AUTO_FOCUS_STATUS, 1);
+			return;
+		case -1:
+			sdl_paint_ui(m);
+			any_write(m->dev);
+			return;
+		}
+		break;
+	}
+			
+	if (event.button.x > m->wx-m->bx)
+		ax = 1;
+	if (event.button.x < m->bx)
+		ax = -1;
+
+	if (event.button.y > m->wy-m->by)
+		ay = 1;
+	if (event.button.y < m->by)
+		ay = -1;
+	    
+	printf("mouse button at...%d, %d area %d, %d\n", event.button.x, event.button.y,
+	       ax, ay);
+
+	int bx = event.button.x - m->bx;
+	int by = event.button.y - m->by;
+
+	/*
+	  AF    |  Focus   |  Aexp         
+	  -------------------------------
+	  ISO   |          |  time         
+	  -------------------------------
+	  Quit  |Fl | nF|F!|  Shoot
+	*/
+	if ((ax == 0) && (ay == -1)) {
+		cam_set_focus(m, ((float) bx)/m->sx);
+		return;
+	}
+	if ((ax == -1) && (ay == 0)) {
+		m->do_exposure = 0;
+		v4l2_set_control(m->fd, 0x00980913, (65536 * by)/m->sy);
+		sdl_sync_settings(m);
+		return;
+	}
+	if ((ax == 1) && (ay == 0)) {
+		m->do_exposure = 0;
+		cam_set_exposure(m, ((double) by)/m->sy);
+		sdl_sync_settings(m);
+		return;
+	}
+	if ((ax == 0) && (ay == 1)) {
+		/* Below */
+	}
+}
+
+static void sdl_iteration(struct sdl *m)
+{
+	SDL_Event event;
+
+	while(SDL_PollEvent(&event)) {
+		switch(event.type) {
+		case SDL_QUIT:
+			exit(1);
+			break;
+		case SDL_KEYDOWN:
+			printf("key pressed... %c\n", event.key.keysym.sym);
+			/* SDLK_A, SDLK_LEFT, SDLK_RETURN, SDLK_BACKSPACE, SDLK_SPACE */
+			switch (event.key.keysym.sym) {
+			default: sdl_key(m, event.key.keysym.sym);
+			}
+			break;
+		case SDL_WINDOWEVENT:
+			if (event.window.event == SDL_WINDOWEVENT_EXPOSED)
+				sdl_paint_ui(m);
+			break;
+		case SDL_MOUSEBUTTONDOWN:
+			sdl_mouse(m, event);
+			break;
+		}
+	}
+}
+
+static void cam_open(struct dev_info *dev)
+{
+	struct v4l2_format *fmt = &dev->fmt;
+
+	dev->fd = v4l2_open("/dev/video0", O_RDWR);
+
+	fmt->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	fmt->fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG10;
+	fmt->fmt.pix.field = V4L2_FIELD_NONE;
+	fmt->fmt.pix.width = 1296;
+	fmt->fmt.pix.height = 984;
+
+	v4l2_s_ctrl(dev->fd, V4L2_CID_AUTO_FOCUS_STATUS, 0);
+	v4l2_set_focus(dev->fd, 50);
+
+	printf("ioctl = %d\n", v4l2_ioctl(dev->fd, VIDIOC_S_FMT, fmt));
+
+	printf("capture is %lx %s %d x %d\n", (unsigned long) fmt->fmt.pix.pixelformat, fmt_name(fmt), fmt->fmt.pix.width, fmt->fmt.pix.height);
+}
+
+/* ------------------------------------------------------------------ */
+
+
+static struct dev_info dev;
+
+int main(void)
+{
+	int i;
+	struct v4l2_format *fmt = &dev.fmt;
+
+	dtime();
+	cam_open(&dev);
+
+	sdl_init(&sdl, &dev, 5);
+  
+	double loop = dtime(), max = 0, avg = .200;
+	if (dev.debug & D_TIMING)
+		printf("startup took %f\n", loop);
+	
+	for (i=0; i<500000; i++) {
+		int num = v4l2_read(dev.fd, dev.buf, SIZE);
+		if (num < 0)
+			return 1;
+
+		{
+			double d = dtime();
+			sdl_render(&sdl, dev.buf, fmt);
+			if (dev.debug & D_TIMING)
+				printf("Render took %f\n", dtime() - d);
+		}
+		{
+			double d = dtime();
+			for (int i = 0; i<1; i++)
+				sdl_status(&sdl, avg);
+			if (dev.debug & D_TIMING)
+				printf("Status took %f\n", dtime() - d);
+		}
+
+		sdl_iteration(&sdl);
+		double now = dtime();
+		if (now - loop > max)
+			max = now - loop;
+		double c = 0.03;
+		avg = (now - loop) * c + avg * (1-c);
+		if (dev.debug & D_TIMING)
+			printf("Iteration %f, maximum %f, average %f\n", now-loop, max, avg);
+		loop = now;
+	}
+	return 0;
+}


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 586 bytes --]

Hi!

> Add simple SDL-based application for capturing photos. Manual
> focus/gain/exposure can be set, flash can be controlled and
> autofocus/autogain can be selected if camera supports that.
> 
> It is already useful for testing autofocus/autogain improvements to
> the libraries on Nokia N900.
> 
> Signed-off-by: Pavel Machek <pavel@ucw.cz>

Could I get some feedback here, or get you to apply the patch?

Thanks,
								Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Hans Verkuil]

On 05/26/2017 10:41 PM, Pavel Machek wrote:
> Hi!
> 
>> Add simple SDL-based application for capturing photos. Manual
>> focus/gain/exposure can be set, flash can be controlled and
>> autofocus/autogain can be selected if camera supports that.
>>
>> It is already useful for testing autofocus/autogain improvements to
>> the libraries on Nokia N900.
>>
>> Signed-off-by: Pavel Machek <pavel@ucw.cz>
> 
> Could I get some feedback here, or get you to apply the patch?

I plan to look at it next week (Monday or Friday).

Regards,

	Hans

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Hans Verkuil]

Hi Pavel,

On 05/21/2017 12:33 PM, Pavel Machek wrote:
> Add simple SDL-based application for capturing photos. Manual
> focus/gain/exposure can be set, flash can be controlled and
> autofocus/autogain can be selected if camera supports that.
> 
> It is already useful for testing autofocus/autogain improvements to
> the libraries on Nokia N900.
> 
> Signed-off-by: Pavel Machek <pavel@ucw.cz>

I think this is more suitable as a github project. To be honest, I feel that
v4l-utils already contains too many random utilities, so I prefer not to add
to that.

On the other hand, there is nothing against sharing this as on github as it
certainly can be useful.

Regards,

	Hans

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1358 bytes --]

On Mon 2017-05-29 08:13:22, Hans Verkuil wrote:
> Hi Pavel,
> 
> On 05/21/2017 12:33 PM, Pavel Machek wrote:
> >Add simple SDL-based application for capturing photos. Manual
> >focus/gain/exposure can be set, flash can be controlled and
> >autofocus/autogain can be selected if camera supports that.
> >
> >It is already useful for testing autofocus/autogain improvements to
> >the libraries on Nokia N900.
> >
> >Signed-off-by: Pavel Machek <pavel@ucw.cz>
> 
> I think this is more suitable as a github project. To be honest, I feel that
> v4l-utils already contains too many random utilities, so I prefer not to add
> to that.

> On the other hand, there is nothing against sharing this as on github as it
> certainly can be useful.

Can I get you to reconsider that?

Originally, I planed to keep the utility separate, but then I got
comments from Mauro ( https://lkml.org/lkml/2017/4/24/457 ) explaining
that hard sdl dependency is not acceptable etc, and how I should do
automake.

So I had a lot of fun with automake integration, and generally doing
things right.

So getting "we all ready have too many utilities" _now_ is quite an
unwelcome surprise.

Regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Hans Verkuil]

On 05/29/2017 09:32 AM, Pavel Machek wrote:
> On Mon 2017-05-29 08:13:22, Hans Verkuil wrote:
>> Hi Pavel,
>>
>> On 05/21/2017 12:33 PM, Pavel Machek wrote:
>>> Add simple SDL-based application for capturing photos. Manual
>>> focus/gain/exposure can be set, flash can be controlled and
>>> autofocus/autogain can be selected if camera supports that.
>>>
>>> It is already useful for testing autofocus/autogain improvements to
>>> the libraries on Nokia N900.
>>>
>>> Signed-off-by: Pavel Machek <pavel@ucw.cz>
>>
>> I think this is more suitable as a github project. To be honest, I feel that
>> v4l-utils already contains too many random utilities, so I prefer not to add
>> to that.
> 
>> On the other hand, there is nothing against sharing this as on github as it
>> certainly can be useful.
> 
> Can I get you to reconsider that?
> 
> Originally, I planed to keep the utility separate, but then I got
> comments from Mauro ( https://lkml.org/lkml/2017/4/24/457 ) explaining
> that hard sdl dependency is not acceptable etc, and how I should do
> automake.
> 
> So I had a lot of fun with automake integration, and generally doing
> things right.
> 
> So getting "we all ready have too many utilities" _now_ is quite an
> unwelcome surprise.

Too many *random* utilities.

Utilities like v4l2-ctl are tied closely to the kernel and are updated whenever
new APIs appear. But yet another viewer?

Mauro, I find that v4l-utils is a bit polluted with non-core utilities.
IMHO it should only contain the core libv4l2, core utilities and driver-specific
utilities. I wonder if we should make a media-utils-contrib for all the non-core
stuff.

What is your opinion?

Regards,

	Hans

> 
> Regards,
> 									Pavel
> 

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Sakari Ailus]

Hi Hans, Pavel, Mauro, others,

On Mon, May 29, 2017 at 10:02:26AM +0200, Hans Verkuil wrote:
> On 05/29/2017 09:32 AM, Pavel Machek wrote:
> >On Mon 2017-05-29 08:13:22, Hans Verkuil wrote:
> >>Hi Pavel,
> >>
> >>On 05/21/2017 12:33 PM, Pavel Machek wrote:
> >>>Add simple SDL-based application for capturing photos. Manual
> >>>focus/gain/exposure can be set, flash can be controlled and
> >>>autofocus/autogain can be selected if camera supports that.
> >>>
> >>>It is already useful for testing autofocus/autogain improvements to
> >>>the libraries on Nokia N900.
> >>>
> >>>Signed-off-by: Pavel Machek <pavel@ucw.cz>
> >>
> >>I think this is more suitable as a github project. To be honest, I feel that
> >>v4l-utils already contains too many random utilities, so I prefer not to add
> >>to that.
> >
> >>On the other hand, there is nothing against sharing this as on github as it
> >>certainly can be useful.
> >
> >Can I get you to reconsider that?
> >
> >Originally, I planed to keep the utility separate, but then I got
> >comments from Mauro ( https://lkml.org/lkml/2017/4/24/457 ) explaining
> >that hard sdl dependency is not acceptable etc, and how I should do
> >automake.
> >
> >So I had a lot of fun with automake integration, and generally doing
> >things right.
> >
> >So getting "we all ready have too many utilities" _now_ is quite an
> >unwelcome surprise.
> 
> Too many *random* utilities.
> 
> Utilities like v4l2-ctl are tied closely to the kernel and are updated whenever
> new APIs appear. But yet another viewer?
> 
> Mauro, I find that v4l-utils is a bit polluted with non-core utilities.
> IMHO it should only contain the core libv4l2, core utilities and driver-specific
> utilities. I wonder if we should make a media-utils-contrib for all the non-core
> stuff.
> 
> What is your opinion?

One of the purposes the v4l-utils repository has is that the distributions
get these programs included to their v4l-utils package as it's typically
called. It's debatable whether or how much it should contain device specific
or otherwise random projects, but having a common location for such programs
has clear benefits, too.

Based on how this one looks it is definitely not an end user application (I
hope I'm not miscategorising it) and as Pavel mentioned, it has been useful
in testing automatic focus / gain control on N900.

Just my 5 euro cents...

-- 
Regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1567 bytes --]

Hi!

> > Utilities like v4l2-ctl are tied closely to the kernel and are updated whenever
> > new APIs appear. But yet another viewer?
> > 
> > Mauro, I find that v4l-utils is a bit polluted with non-core utilities.
> > IMHO it should only contain the core libv4l2, core utilities and driver-specific
> > utilities. I wonder if we should make a media-utils-contrib for all the non-core
> > stuff.
> > 
> > What is your opinion?
> 
> One of the purposes the v4l-utils repository has is that the distributions
> get these programs included to their v4l-utils package as it's typically
> called. It's debatable whether or how much it should contain device specific
> or otherwise random projects, but having a common location for such programs
> has clear benefits, too.
> 
> Based on how this one looks it is definitely not an end user application (I
> hope I'm not miscategorising it) and as Pavel mentioned, it has been useful
> in testing automatic focus / gain control on N900.

Well, I intend to take some photos with it. But yes, it is more useful
for testing -- it is hard to test digital camera without taking photos
-- and eventually I'd like to turn most of it into a library which
"real" camera application would link to.

But that's future. For now I'd like to have something I can test the
kernel drivers with, and perhaps take some photos in the process.

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: add sdlcam example for testing digital still camera functionality

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1502 bytes --]

Hi!

> > Utilities like v4l2-ctl are tied closely to the kernel and are updated whenever
> > new APIs appear. But yet another viewer?
> > 
> > Mauro, I find that v4l-utils is a bit polluted with non-core utilities.
> > IMHO it should only contain the core libv4l2, core utilities and driver-specific
> > utilities. I wonder if we should make a media-utils-contrib for all the non-core
> > stuff.
> > 
> > What is your opinion?
> 
> One of the purposes the v4l-utils repository has is that the distributions
> get these programs included to their v4l-utils package as it's typically
> called. It's debatable whether or how much it should contain device specific
> or otherwise random projects, but having a common location for such programs
> has clear benefits, too.
> 
> Based on how this one looks it is definitely not an end user application (I
> hope I'm not miscategorising it) and as Pavel mentioned, it has been useful
> in testing automatic focus / gain control on N900.

Well, testing camera drivers without userspace support is not really
possible. Yes, you can use mplayer to display the data in "preview"
mode, and yavta to adjust focus/exposure... But that's really
hard/slow and preview quality does not tell you much about quality of
final photo.

So... yes, I'd like to see this in.

Best regards,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch, libv4l]: fix integer overflow

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1198 bytes --]


Fix integer overflow with EXPOSURE_ABSOLUTE. This is problem for
example with Nokia N900.

Signed-off-by: Pavel Machek <pavel@ucw.cz>

diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index e795aee..189fc06 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1776,7 +1776,7 @@ int v4l2_set_control(int fd, int cid, int value)
 		if (qctrl.type == V4L2_CTRL_TYPE_BOOLEAN)
 			ctrl.value = value ? 1 : 0;
 		else
-			ctrl.value = (value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
+			ctrl.value = ((long long) value * (qctrl.maximum - qctrl.minimum) + 32767) / 65535 +
 				qctrl.minimum;
 
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, &ctrl);
@@ -1812,7 +1812,7 @@ int v4l2_get_control(int fd, int cid)
 		if (v4l2_propagate_ioctl(index, VIDIOC_G_CTRL, &ctrl))
 			return -1;
 
-	return ((ctrl.value - qctrl.minimum) * 65535 +
+	return (((long long) ctrl.value - qctrl.minimum) * 65535 +
 			(qctrl.maximum - qctrl.minimum) / 2) /
 		(qctrl.maximum - qctrl.minimum);
 }





-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 5135 bytes --]

On Wed 2017-04-26 15:23:37, Pavel Machek wrote:
> Hi!
> 
> > > > I don't see why it would be hard to open files or have threads inside
> > > > a library. There are several libraries that do that already, specially
> > > > the ones designed to be used on multimidia apps.  
> > > 
> > > Well, This is what the libv4l2 says:
> > > 
> > >    This file implements libv4l2, which offers v4l2_ prefixed versions
> > >    of
> > >       open/close/etc. The API is 100% the same as directly opening
> > >    /dev/videoX
> > >       using regular open/close/etc, the big difference is that format
> > >    conversion
> > >    
> > > but if I open additional files in v4l2_open(), API is no longer the
> > > same, as unix open() is defined to open just one file descriptor.
> > > 
> > > Now. There is autogain support in libv4lconvert, but it expects to use
> > > same fd for camera and for the gain... which does not work with
> > > subdevs.
> > > 
> > > Of course, opening subdevs by name like this is not really
> > > acceptable. But can you suggest a method that is?
> > 
> > There are two separate things here:
> > 
> > 1) Autofoucs for a device that doesn't use subdev API
> > 2) libv4l2 support for devices that require MC and subdev API
> 
> Actually there are three: 0) autogain. Unfortunately, I need autogain
> first before autofocus has a chance...
> 
> And that means... bayer10 support for autogain.
> 
> Plus, I changed avg_lum to long long. Quick calculation tells me int
> could overflow with few megapixel sensor.
> 
> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> longer works.

Can I get some comments here? Patch will need fixup (constants need
adjusting), but is style/design acceptable?

Thanks,
								Pavel

> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> index c6866d6..0b52d0f 100644
> --- a/lib/libv4lconvert/processing/autogain.c
> +++ b/lib/libv4lconvert/processing/autogain.c
> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>  	}
>  }
>  
> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	avg_lum /= 4;
> +	return avg_lum;
> +}
> +
> +static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	return avg_lum;
> +}
> +
>  /* auto gain and exposure algorithm based on the knee algorithm described here:
>  http://ytse.tricolour.net/docs/LowLightOptimization.html */
>  static int autogain_calculate_lookup_tables(
> @@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
>  	switch (fmt->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
> +		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
> +		break;
> +
>  	case V4L2_PIX_FMT_SGBRG8:
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> -		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> -			fmt->fmt.pix.width / 4;
> -
> -		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> -			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> -				avg_lum += *buf++;
> -			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> -		}
> -		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +		avg_lum = get_luminosity_bayer8(buf, fmt);
>  		break;
>  
>  	case V4L2_PIX_FMT_RGB24:
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
> index b061f50..b98d024 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing.c
> +++ b/lib/libv4lconvert/processing/libv4lprocessing.c
> @@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
>  	case V4L2_PIX_FMT_RGB24:
>  	case V4L2_PIX_FMT_BGR24:
>  		break;
> 
> 
> 



-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] autogain support for bayer10 format (was Re: [patch] propagating controls in libv4l2)

[Pavel Machek]

On Wed 2017-04-26 15:23:37, Pavel Machek wrote:
> Hi!
> 
> > > > I don't see why it would be hard to open files or have threads inside
> > > > a library. There are several libraries that do that already, specially
> > > > the ones designed to be used on multimidia apps.  
> > > 
> > > Well, This is what the libv4l2 says:
> > > 
> > >    This file implements libv4l2, which offers v4l2_ prefixed versions
> > >    of
> > >       open/close/etc. The API is 100% the same as directly opening
> > >    /dev/videoX
> > >       using regular open/close/etc, the big difference is that format
> > >    conversion
> > >    
> > > but if I open additional files in v4l2_open(), API is no longer the
> > > same, as unix open() is defined to open just one file descriptor.
> > > 
> > > Now. There is autogain support in libv4lconvert, but it expects to use
> > > same fd for camera and for the gain... which does not work with
> > > subdevs.
> > > 
> > > Of course, opening subdevs by name like this is not really
> > > acceptable. But can you suggest a method that is?
> > 
> > There are two separate things here:
> > 
> > 1) Autofoucs for a device that doesn't use subdev API
> > 2) libv4l2 support for devices that require MC and subdev API
> 
> Actually there are three: 0) autogain. Unfortunately, I need autogain
> first before autofocus has a chance...
> 
> And that means... bayer10 support for autogain.
> 
> Plus, I changed avg_lum to long long. Quick calculation tells me int
> could overflow with few megapixel sensor.
> 
> Oh, btw http://ytse.tricolour.net/docs/LowLightOptimization.html no
> longer works.

Can I get some comments here? Patch will need fixup (constants need
adjusting), but is style/design acceptable?

Thanks,
								Pavel

> diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
> index c6866d6..0b52d0f 100644
> --- a/lib/libv4lconvert/processing/autogain.c
> +++ b/lib/libv4lconvert/processing/autogain.c
> @@ -68,6 +71,41 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
>  	}
>  }
>  
> +static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	avg_lum /= 4;
> +	return avg_lum;
> +}
> +
> +static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
> +{
> +	long long avg_lum = 0;
> +	int x, y;
> +	
> +	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> +		fmt->fmt.pix.width / 4;
> +
> +	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> +		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> +			avg_lum += *buf++;
> +		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> +	}
> +	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +	return avg_lum;
> +}
> +
>  /* auto gain and exposure algorithm based on the knee algorithm described here:
>  http://ytse.tricolour.net/docs/LowLightOptimization.html */
>  static int autogain_calculate_lookup_tables(
> @@ -100,17 +142,16 @@ static int autogain_calculate_lookup_tables(
>  	switch (fmt->fmt.pix.pixelformat) {
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
> +		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
> +		break;
> +
>  	case V4L2_PIX_FMT_SGBRG8:
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> -		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
> -			fmt->fmt.pix.width / 4;
> -
> -		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
> -			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
> -				avg_lum += *buf++;
> -			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
> -		}
> -		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
> +		avg_lum = get_luminosity_bayer8(buf, fmt);
>  		break;
>  
>  	case V4L2_PIX_FMT_RGB24:
> diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
> index b061f50..b98d024 100644
> --- a/lib/libv4lconvert/processing/libv4lprocessing.c
> +++ b/lib/libv4lconvert/processing/libv4lprocessing.c
> @@ -164,6 +165,10 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
>  	case V4L2_PIX_FMT_SGRBG8:
>  	case V4L2_PIX_FMT_SBGGR8:
>  	case V4L2_PIX_FMT_SRGGB8:
> +	case V4L2_PIX_FMT_SGBRG10:
> +	case V4L2_PIX_FMT_SGRBG10:
> +	case V4L2_PIX_FMT_SBGGR10:
> +	case V4L2_PIX_FMT_SRGGB10:
>  	case V4L2_PIX_FMT_RGB24:
>  	case V4L2_PIX_FMT_BGR24:
>  		break;
> 
> 
> 



-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170501/f729b3a9/attachment-0001.sig>

Re: [patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 6487 bytes --]

Hi!

> > Now. There is autogain support in libv4lconvert, but it expects to use
> > same fd for camera and for the gain... which does not work with
> > subdevs.
> > 
> > Of course, opening subdevs by name like this is not really
> > acceptable. But can you suggest a method that is?
> 
> There are two separate things here:
> 
> 1) Autofoucs for a device that doesn't use subdev API
> 2) libv4l2 support for devices that require MC and subdev API
> 
> for (1), it should use the /dev/videoX device that was opened with
> v4l2_open().
> 
> For (2), libv4l2 should be aware of MC and subdev APIs. Sakari
> once tried to write a libv4l2 plugin for OMAP3, but never finished it.
> A more recent trial were to add a libv4l2 plugin for Exynos.
> Unfortunately, none of those code got merged. Last time I checked,
> the Exynos plugin was almost ready to be merged, but Sakari asked
> some changes on it. The developer that was working on it got job on
> some other company. Last time I heard from him, he was still interested
> on finishing his work, but in the need to setup a test environment
> using his own devices.

First... we should not have hardware-specific code in the
userspace. Hardware abstraction should be kernel's job.

Second... we really should solve "ioctl propagation" in
libv4l2. Because otherwise existing userspace is unusable in MC/subdev
drivers.

> IMHO, the right thing to do with regards to autofocus is to
> implement it via a processing module, assuming that just one
> video device is opened.

Ok, I have done that.

> The hole idea is that a libv4l2 client, running on a N900 device
> would just open a fake /dev/video0. Internally, libv4l2 will
> open whatever video nodes it needs to control the device, exporting
> all hardware capabilities (video formats, controls, resolutions,
> etc) as if it was a normal V4L2 camera, hiding all dirty details
> about MC and subdev APIs from userspace application.
> 
> This way, a normal application, like xawtv, tvtime, camorama,
> zbar, mplayer, vlc, ... will work without any changes.

Well, yes, we'd like to get there eventually. But we are not there at
the moment, and ioctl() propagation is one of the steps.

(I do have support specific for N900, but currently it is in python;
this is enough to make the camera usable.)

Regards,
								Pavel

								
> > diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> > index 343db5e..a6bc48e 100644
> > --- a/lib/libv4l2/libv4l2-priv.h
> > +++ b/lib/libv4l2/libv4l2-priv.h
> > @@ -26,6 +26,7 @@
> >  #include "../libv4lconvert/libv4lsyscall-priv.h"
> >  
> >  #define V4L2_MAX_DEVICES 16
> > +#define V4L2_MAX_SUBDEVS 8
> >  /* Warning when making this larger the frame_queued and frame_mapped members of
> >     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
> >     be adjusted! */
> > @@ -104,6 +105,7 @@ struct v4l2_dev_info {
> >  	void *plugin_library;
> >  	void *dev_ops_priv;
> >  	const struct libv4l_dev_ops *dev_ops;
> > +        int subdev_fds[V4L2_MAX_SUBDEVS];
> >  };
> >  
> >  /* From v4l2-plugin.c */
> > diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> > index 0ba0a88..edc9642 100644
> > --- a/lib/libv4l2/libv4l2.c
> > +++ b/lib/libv4l2/libv4l2.c
> > @@ -1,3 +1,4 @@
> > +/* -*- c-file-style: "linux" -*- */
> >  /*
> >  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
> >  
> > @@ -789,18 +790,25 @@ no_capture:
> >  
> >  	/* Note we always tell v4lconvert to optimize src fmt selection for
> >  	   our default fps, the only exception is the app explicitly selecting
> > -	   a fram erate using the S_PARM ioctl after a S_FMT */
> > +	   a frame rate using the S_PARM ioctl after a S_FMT */
> >  	if (devices[index].convert)
> >  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
> >  	v4l2_update_fps(index, &parm);
> >  
> > +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> > +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> > +	devices[index].subdev_fds[2] = -1;
> > +
> > +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> > +	       devices[index].subdev_fds[1]);
> > +
> >  	V4L2_LOG("open: %d\n", fd);
> >  
> >  	return fd;
> >  }
> >  
> >  /* Is this an fd for which we are emulating v4l1 ? */
> > -static int v4l2_get_index(int fd)
> > +int v4l2_get_index(int fd)
> >  {
> >  	int index;
> >  
> > 
> > commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
> > Author: Pavel <pavel@ucw.cz>
> > Date:   Wed Apr 26 12:34:04 2017 +0200
> > 
> >     Enable ioctl propagation.
> > 
> > diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> > index edc9642..6dab661 100644
> > --- a/lib/libv4l2/libv4l2.c
> > +++ b/lib/libv4l2/libv4l2.c
> > @@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
> >  	return 0;
> >  }
> >  
> > +static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
> > +{
> > +	int i = 0;
> > +	int result;
> > +	while (1) {
> > +		if (devices[index].subdev_fds[i] == -1)
> > +			return -1;
> > +		printf("g_ctrl failed, trying...\n");
> > +		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
> > +		printf("subdev %d result %d\n", i, result);
> > +		if (result == 0)
> > +			return 0;
> > +		i++;
> > +	}
> > +	return -1;
> > +}
> > +
> >  int v4l2_ioctl(int fd, unsigned long int request, ...)
> >  {
> >  	void *arg;
> > @@ -1193,14 +1210,20 @@ no_capture_request:
> >  	switch (request) {
> >  	case VIDIOC_QUERYCTRL:
> >  		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_G_CTRL:
> >  		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_S_CTRL:
> >  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_G_EXT_CTRLS:
> > 
> > 
> 
> 
> 
> Thanks,
> Mauro

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > Now. There is autogain support in libv4lconvert, but it expects to use
> > same fd for camera and for the gain... which does not work with
> > subdevs.
> > 
> > Of course, opening subdevs by name like this is not really
> > acceptable. But can you suggest a method that is?
> 
> There are two separate things here:
> 
> 1) Autofoucs for a device that doesn't use subdev API
> 2) libv4l2 support for devices that require MC and subdev API
> 
> for (1), it should use the /dev/videoX device that was opened with
> v4l2_open().
> 
> For (2), libv4l2 should be aware of MC and subdev APIs. Sakari
> once tried to write a libv4l2 plugin for OMAP3, but never finished it.
> A more recent trial were to add a libv4l2 plugin for Exynos.
> Unfortunately, none of those code got merged. Last time I checked,
> the Exynos plugin was almost ready to be merged, but Sakari asked
> some changes on it. The developer that was working on it got job on
> some other company. Last time I heard from him, he was still interested
> on finishing his work, but in the need to setup a test environment
> using his own devices.

First... we should not have hardware-specific code in the
userspace. Hardware abstraction should be kernel's job.

Second... we really should solve "ioctl propagation" in
libv4l2. Because otherwise existing userspace is unusable in MC/subdev
drivers.

> IMHO, the right thing to do with regards to autofocus is to
> implement it via a processing module, assuming that just one
> video device is opened.

Ok, I have done that.

> The hole idea is that a libv4l2 client, running on a N900 device
> would just open a fake /dev/video0. Internally, libv4l2 will
> open whatever video nodes it needs to control the device, exporting
> all hardware capabilities (video formats, controls, resolutions,
> etc) as if it was a normal V4L2 camera, hiding all dirty details
> about MC and subdev APIs from userspace application.
> 
> This way, a normal application, like xawtv, tvtime, camorama,
> zbar, mplayer, vlc, ... will work without any changes.

Well, yes, we'd like to get there eventually. But we are not there at
the moment, and ioctl() propagation is one of the steps.

(I do have support specific for N900, but currently it is in python;
this is enough to make the camera usable.)

Regards,
								Pavel

								
> > diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> > index 343db5e..a6bc48e 100644
> > --- a/lib/libv4l2/libv4l2-priv.h
> > +++ b/lib/libv4l2/libv4l2-priv.h
> > @@ -26,6 +26,7 @@
> >  #include "../libv4lconvert/libv4lsyscall-priv.h"
> >  
> >  #define V4L2_MAX_DEVICES 16
> > +#define V4L2_MAX_SUBDEVS 8
> >  /* Warning when making this larger the frame_queued and frame_mapped members of
> >     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
> >     be adjusted! */
> > @@ -104,6 +105,7 @@ struct v4l2_dev_info {
> >  	void *plugin_library;
> >  	void *dev_ops_priv;
> >  	const struct libv4l_dev_ops *dev_ops;
> > +        int subdev_fds[V4L2_MAX_SUBDEVS];
> >  };
> >  
> >  /* From v4l2-plugin.c */
> > diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> > index 0ba0a88..edc9642 100644
> > --- a/lib/libv4l2/libv4l2.c
> > +++ b/lib/libv4l2/libv4l2.c
> > @@ -1,3 +1,4 @@
> > +/* -*- c-file-style: "linux" -*- */
> >  /*
> >  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
> >  
> > @@ -789,18 +790,25 @@ no_capture:
> >  
> >  	/* Note we always tell v4lconvert to optimize src fmt selection for
> >  	   our default fps, the only exception is the app explicitly selecting
> > -	   a fram erate using the S_PARM ioctl after a S_FMT */
> > +	   a frame rate using the S_PARM ioctl after a S_FMT */
> >  	if (devices[index].convert)
> >  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
> >  	v4l2_update_fps(index, &parm);
> >  
> > +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> > +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> > +	devices[index].subdev_fds[2] = -1;
> > +
> > +	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
> > +	       devices[index].subdev_fds[1]);
> > +
> >  	V4L2_LOG("open: %d\n", fd);
> >  
> >  	return fd;
> >  }
> >  
> >  /* Is this an fd for which we are emulating v4l1 ? */
> > -static int v4l2_get_index(int fd)
> > +int v4l2_get_index(int fd)
> >  {
> >  	int index;
> >  
> > 
> > commit 1d6a9ce121f53e8f2e38549eed597a3c3dea5233
> > Author: Pavel <pavel@ucw.cz>
> > Date:   Wed Apr 26 12:34:04 2017 +0200
> > 
> >     Enable ioctl propagation.
> > 
> > diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> > index edc9642..6dab661 100644
> > --- a/lib/libv4l2/libv4l2.c
> > +++ b/lib/libv4l2/libv4l2.c
> > @@ -1064,6 +1064,23 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
> >  	return 0;
> >  }
> >  
> > +static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
> > +{
> > +	int i = 0;
> > +	int result;
> > +	while (1) {
> > +		if (devices[index].subdev_fds[i] == -1)
> > +			return -1;
> > +		printf("g_ctrl failed, trying...\n");
> > +		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
> > +		printf("subdev %d result %d\n", i, result);
> > +		if (result == 0)
> > +			return 0;
> > +		i++;
> > +	}
> > +	return -1;
> > +}
> > +
> >  int v4l2_ioctl(int fd, unsigned long int request, ...)
> >  {
> >  	void *arg;
> > @@ -1193,14 +1210,20 @@ no_capture_request:
> >  	switch (request) {
> >  	case VIDIOC_QUERYCTRL:
> >  		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_G_CTRL:
> >  		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_S_CTRL:
> >  		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
> > +		if (result == -1)
> > +			result = v4l2_propagate_ioctl(index, request, arg);
> >  		break;
> >  
> >  	case VIDIOC_G_EXT_CTRLS:
> > 
> > 
> 
> 
> 
> Thanks,
> Mauro

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170713/72bee76f/attachment.sig>

Re: [patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Wed, 26 Apr 2017 12:53:00 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > > separate "video server" application looks really weird for me.    
> > > 
> > > Well... let me see. libraries are quite limited -- it is hard to open
> > > files, or use threads/have custom main loop. It may be useful to
> > > switch resolutions -- do autofocus/autogain at lower resolution, then
> > > switch to high one for taking picture. It would be good to have that
> > > in "system" code, but I'm not at all sure libv4l2 design will allow
> > > that.  
> > 
> > I don't see why it would be hard to open files or have threads inside
> > a library. There are several libraries that do that already, specially
> > the ones designed to be used on multimidia apps.  
> 
> Well, This is what the libv4l2 says:
> 
>    This file implements libv4l2, which offers v4l2_ prefixed versions
>    of
>       open/close/etc. The API is 100% the same as directly opening
>    /dev/videoX
>       using regular open/close/etc, the big difference is that format
>    conversion
>    
> but if I open additional files in v4l2_open(), API is no longer the
> same, as unix open() is defined to open just one file descriptor.
> 
> Now. There is autogain support in libv4lconvert, but it expects to use
> same fd for camera and for the gain... which does not work with
> subdevs.
> 
> Of course, opening subdevs by name like this is not really
> acceptable. But can you suggest a method that is?
> 
> Thanks,
> 								Pavel
> 
> commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 11:38:04 2017 +0200
> 
>     Add subdevices.
> 
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..a6bc48e 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -26,6 +26,7 @@
>  #include "../libv4lconvert/libv4lsyscall-priv.h"
>  
>  #define V4L2_MAX_DEVICES 16
> +#define V4L2_MAX_SUBDEVS 8

Isn't it a short number?

>  /* Warning when making this larger the frame_queued and frame_mapped members of
>     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
>     be adjusted! */
> @@ -104,6 +105,7 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..edc9642 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */

No emacs comments, please.

>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -789,18 +790,25 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +	devices[index].subdev_fds[2] = -1;

Hardcoding names here is not a good idea. Ideally, it should open
the MC, using the newgen API, and parse the media graph.

The problem is that, even with newgen API, without the properties API
you likely won't be able to write a generic parser. So, we need a
plugin specific for OMAP3 (or at least some database that would teach
a generic plugin about OMAP3 specifics).

I guess that the approach that Jacek was taken were very close to what
a generic plugin would need:
	https://lwn.net/Articles/619449/

The last version of his patch set is here:
	https://patchwork.linuxtv.org/patch/37496/

I didn't review his patchset, but from what I saw, Sakari is the one
that found some issues on v7.1 patchset.

Sakari,

Could you shed us a light about why this patchset was not merged?

Are there anything really bad at the code, or just minor issues that
could be fixed later?

If it is the last case, perhaps we could merge the code, if this
would make easier for Pavel to work on a N9 solution using the
same approach.


Thanks,
Mauro

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Wed, 26 Apr 2017 12:53:00 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > > > IMO, the best place for autofocus is at libv4l2. Putting it on a
> > > > separate "video server" application looks really weird for me.    
> > > 
> > > Well... let me see. libraries are quite limited -- it is hard to open
> > > files, or use threads/have custom main loop. It may be useful to
> > > switch resolutions -- do autofocus/autogain at lower resolution, then
> > > switch to high one for taking picture. It would be good to have that
> > > in "system" code, but I'm not at all sure libv4l2 design will allow
> > > that.  
> > 
> > I don't see why it would be hard to open files or have threads inside
> > a library. There are several libraries that do that already, specially
> > the ones designed to be used on multimidia apps.  
> 
> Well, This is what the libv4l2 says:
> 
>    This file implements libv4l2, which offers v4l2_ prefixed versions
>    of
>       open/close/etc. The API is 100% the same as directly opening
>    /dev/videoX
>       using regular open/close/etc, the big difference is that format
>    conversion
>    
> but if I open additional files in v4l2_open(), API is no longer the
> same, as unix open() is defined to open just one file descriptor.
> 
> Now. There is autogain support in libv4lconvert, but it expects to use
> same fd for camera and for the gain... which does not work with
> subdevs.
> 
> Of course, opening subdevs by name like this is not really
> acceptable. But can you suggest a method that is?
> 
> Thanks,
> 								Pavel
> 
> commit 4cf9d10ead014c0db25452e4bb9cd144632407c3
> Author: Pavel <pavel@ucw.cz>
> Date:   Wed Apr 26 11:38:04 2017 +0200
> 
>     Add subdevices.
> 
> diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
> index 343db5e..a6bc48e 100644
> --- a/lib/libv4l2/libv4l2-priv.h
> +++ b/lib/libv4l2/libv4l2-priv.h
> @@ -26,6 +26,7 @@
>  #include "../libv4lconvert/libv4lsyscall-priv.h"
>  
>  #define V4L2_MAX_DEVICES 16
> +#define V4L2_MAX_SUBDEVS 8

Isn't it a short number?

>  /* Warning when making this larger the frame_queued and frame_mapped members of
>     the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
>     be adjusted! */
> @@ -104,6 +105,7 @@ struct v4l2_dev_info {
>  	void *plugin_library;
>  	void *dev_ops_priv;
>  	const struct libv4l_dev_ops *dev_ops;
> +        int subdev_fds[V4L2_MAX_SUBDEVS];
>  };
>  
>  /* From v4l2-plugin.c */
> diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
> index 0ba0a88..edc9642 100644
> --- a/lib/libv4l2/libv4l2.c
> +++ b/lib/libv4l2/libv4l2.c
> @@ -1,3 +1,4 @@
> +/* -*- c-file-style: "linux" -*- */

No emacs comments, please.

>  /*
>  #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
>  
> @@ -789,18 +790,25 @@ no_capture:
>  
>  	/* Note we always tell v4lconvert to optimize src fmt selection for
>  	   our default fps, the only exception is the app explicitly selecting
> -	   a fram erate using the S_PARM ioctl after a S_FMT */
> +	   a frame rate using the S_PARM ioctl after a S_FMT */
>  	if (devices[index].convert)
>  		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
>  	v4l2_update_fps(index, &parm);
>  
> +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> +	devices[index].subdev_fds[2] = -1;

Hardcoding names here is not a good idea. Ideally, it should open
the MC, using the newgen API, and parse the media graph.

The problem is that, even with newgen API, without the properties API
you likely won't be able to write a generic parser. So, we need a
plugin specific for OMAP3 (or at least some database that would teach
a generic plugin about OMAP3 specifics).

I guess that the approach that Jacek was taken were very close to what
a generic plugin would need:
	https://lwn.net/Articles/619449/

The last version of his patch set is here:
	https://patchwork.linuxtv.org/patch/37496/

I didn't review his patchset, but from what I saw, Sakari is the one
that found some issues on v7.1 patchset.

Sakari,

Could you shed us a light about why this patchset was not merged?

Are there anything really bad at the code, or just minor issues that
could be fixed later?

If it is the last case, perhaps we could merge the code, if this
would make easier for Pavel to work on a N9 solution using the
same approach.


Thanks,
Mauro

Re: [patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1576 bytes --]

Hi!

> > +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> > +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> > +	devices[index].subdev_fds[2] = -1;
> 
> Hardcoding names here is not a good idea. Ideally, it should open
> the MC, using the newgen API, and parse the media graph.
> 
> The problem is that, even with newgen API, without the properties API
> you likely won't be able to write a generic parser. So, we need a
> plugin specific for OMAP3 (or at least some database that would teach
> a generic plugin about OMAP3 specifics).
> 
> I guess that the approach that Jacek was taken were very close to what
> a generic plugin would need:
> 	https://lwn.net/Articles/619449/
> 
> The last version of his patch set is here:
> 	https://patchwork.linuxtv.org/patch/37496/
> 
> I didn't review his patchset, but from what I saw, Sakari is the one
> that found some issues on v7.1 patchset.
> 
> Sakari,
> 
> Could you shed us a light about why this patchset was not merged?
> 
> Are there anything really bad at the code, or just minor issues that
> could be fixed later?
> 
> If it is the last case, perhaps we could merge the code, if this
> would make easier for Pavel to work on a N9 solution using the
> same approach.

It would be nice to get some solution here. Camera without libv4l
support is pretty much useless :-(.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[patch] propagating controls in libv4l2 was Re: support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > +	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
> > +	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
> > +	devices[index].subdev_fds[2] = -1;
> 
> Hardcoding names here is not a good idea. Ideally, it should open
> the MC, using the newgen API, and parse the media graph.
> 
> The problem is that, even with newgen API, without the properties API
> you likely won't be able to write a generic parser. So, we need a
> plugin specific for OMAP3 (or at least some database that would teach
> a generic plugin about OMAP3 specifics).
> 
> I guess that the approach that Jacek was taken were very close to what
> a generic plugin would need:
> 	https://lwn.net/Articles/619449/
> 
> The last version of his patch set is here:
> 	https://patchwork.linuxtv.org/patch/37496/
> 
> I didn't review his patchset, but from what I saw, Sakari is the one
> that found some issues on v7.1 patchset.
> 
> Sakari,
> 
> Could you shed us a light about why this patchset was not merged?
> 
> Are there anything really bad at the code, or just minor issues that
> could be fixed later?
> 
> If it is the last case, perhaps we could merge the code, if this
> would make easier for Pavel to work on a N9 solution using the
> same approach.

It would be nice to get some solution here. Camera without libv4l
support is pretty much useless :-(.

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170429/5a448afb/attachment.sig>

Camera support, Prague next week, sdlcam

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 92623 bytes --]

Hi!

I'd still like to get some reasonable support for cellphone camera in
Linux.

IMO first reasonable step is to merge sdlcam, then we can implement
autofocus, improve autogain... and rest of the boring stuff. Ouch and
media graph support would be nice. Currently, _nothing_ works with
media graph device, such as N900.

I'll talk about the issues at ELCE in few days:

https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh

Will someone else be there? Is there some place where v4l people meet?

Best regards,

								Pavel

commit 8db1546cd0b5229ad7bf2605fd5c295365df57f8
Author: Pavel <pavel@ucw.cz>
Date:   Fri Oct 13 21:24:16 2017 +0200

    Port changes from "cleaner" branch. For some reason, SDL2_image is
    missing in compilation.
    
    gcc -std=gnu99 -I../.. -g -O2 -o sdlcam sdlcam-sdlcam.o  -ljpeg -lSDL2
    -lSDL2_image ../../lib/libv4l2/.libs/libv4l2.a ../../lib/libv4lconvert/.libs/libv4lconvert.a

diff --git a/configure.ac b/configure.ac
index f3691be..dc412c9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -98,6 +98,247 @@ LIBDVBV5_DOMAIN="libdvbv5"
 AC_DEFINE([LIBDVBV5_DOMAIN], "libdvbv5", [libdvbv5 domain])
 AC_SUBST(LIBDVBV5_DOMAIN)
 
+# Configure paths for SDL
+# Sam Lantinga 9/21/99
+# stolen from Manish Singh
+# stolen back from Frank Belew
+# stolen from Manish Singh
+# Shamelessly stolen from Owen Taylor
+#
+# Changelog:
+# * also look for SDL2.framework under Mac OS X
+
+# serial 1
+
+dnl AM_PATH_SDL2([MINIMUM-VERSION, [ACTION-IF-FOUND [, ACTION-IF-NOT-FOUND]]])
+dnl Test for SDL, and define SDL_CFLAGS and SDL_LIBS
+dnl
+AC_DEFUN([AM_PATH_SDL2],
+[dnl 
+dnl Get the cflags and libraries from the sdl2-config script
+dnl
+AC_ARG_WITH(sdl-prefix,[  --with-sdl-prefix=PFX   Prefix where SDL is installed (optional)],
+            sdl_prefix="$withval", sdl_prefix="")
+AC_ARG_WITH(sdl-exec-prefix,[  --with-sdl-exec-prefix=PFX Exec prefix where SDL is installed (optional)],
+            sdl_exec_prefix="$withval", sdl_exec_prefix="")
+AC_ARG_ENABLE(sdltest, [  --disable-sdltest       Do not try to compile and run a test SDL program],
+		    , enable_sdltest=yes)
+AC_ARG_ENABLE(sdlframework, [  --disable-sdlframework Do not search for SDL2.framework],
+        , search_sdl_framework=yes)
+
+AC_ARG_VAR(SDL2_FRAMEWORK, [Path to SDL2.framework])
+
+  min_sdl_version=ifelse([$1], ,2.0.0,$1)
+
+  if test "x$sdl_prefix$sdl_exec_prefix" = x ; then
+    PKG_CHECK_MODULES([SDL], [sdl2 >= $min_sdl_version],
+           [sdl_pc=yes],
+           [sdl_pc=no])
+  else
+    sdl_pc=no
+    if test x$sdl_exec_prefix != x ; then
+      sdl_config_args="$sdl_config_args --exec-prefix=$sdl_exec_prefix"
+      if test x${SDL2_CONFIG+set} != xset ; then
+        SDL2_CONFIG=$sdl_exec_prefix/bin/sdl2-config
+      fi
+    fi
+    if test x$sdl_prefix != x ; then
+      sdl_config_args="$sdl_config_args --prefix=$sdl_prefix"
+      if test x${SDL2_CONFIG+set} != xset ; then
+        SDL2_CONFIG=$sdl_prefix/bin/sdl2-config
+      fi
+    fi
+  fi
+
+  if test "x$sdl_pc" = xyes ; then
+    no_sdl=""
+    SDL2_CONFIG="pkg-config sdl2"
+  else
+    as_save_PATH="$PATH"
+    if test "x$prefix" != xNONE && test "$cross_compiling" != yes; then
+      PATH="$prefix/bin:$prefix/usr/bin:$PATH"
+    fi
+    AC_PATH_PROG(SDL2_CONFIG, sdl2-config, no, [$PATH])
+    PATH="$as_save_PATH"
+    no_sdl=""
+
+    if test "$SDL2_CONFIG" = "no" -a "x$search_sdl_framework" = "xyes"; then
+      AC_MSG_CHECKING(for SDL2.framework)
+      if test "x$SDL2_FRAMEWORK" != x; then
+        sdl_framework=$SDL2_FRAMEWORK
+      else
+        for d in / ~/ /System/; do
+          if test -d "$dLibrary/Frameworks/SDL2.framework"; then
+            sdl_framework="$dLibrary/Frameworks/SDL2.framework"
+          fi
+        done
+      fi
+
+      if test -d $sdl_framework; then
+        AC_MSG_RESULT($sdl_framework)
+        sdl_framework_dir=`dirname $sdl_framework`
+        SDL_CFLAGS="-F$sdl_framework_dir -Wl,-framework,SDL2 -I$sdl_framework/include"
+        SDL_LIBS="-F$sdl_framework_dir -Wl,-framework,SDL2"
+      else
+        no_sdl=yes
+      fi
+    fi
+
+    if test "$SDL2_CONFIG" != "no"; then
+      if test "x$sdl_pc" = "xno"; then
+        AC_MSG_CHECKING(for SDL - version >= $min_sdl_version)
+        SDL_CFLAGS=`$SDL2_CONFIG $sdl_config_args --cflags`
+        SDL_LIBS=`$SDL2_CONFIG $sdl_config_args --libs`
+      fi
+
+      sdl_major_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\1/'`
+      sdl_minor_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\2/'`
+      sdl_micro_version=`$SDL2_CONFIG $sdl_config_args --version | \
+             sed 's/\([[0-9]]*\).\([[0-9]]*\).\([[0-9]]*\)/\3/'`
+      if test "x$enable_sdltest" = "xyes" ; then
+        ac_save_CFLAGS="$CFLAGS"
+        ac_save_CXXFLAGS="$CXXFLAGS"
+        ac_save_LIBS="$LIBS"
+        CFLAGS="$CFLAGS $SDL_CFLAGS"
+        CXXFLAGS="$CXXFLAGS $SDL_CFLAGS"
+        LIBS="$LIBS $SDL_LIBS"
+dnl
+dnl Now check if the installed SDL is sufficiently new. (Also sanity
+dnl checks the results of sdl2-config to some extent
+dnl
+      rm -f conf.sdltest
+      AC_TRY_RUN([
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include "SDL.h"
+
+char*
+my_strdup (char *str)
+{
+  char *new_str;
+  
+  if (str)
+    {
+      new_str = (char *)malloc ((strlen (str) + 1) * sizeof(char));
+      strcpy (new_str, str);
+    }
+  else
+    new_str = NULL;
+  
+  return new_str;
+}
+
+int main (int argc, char *argv[])
+{
+  int major, minor, micro;
+  char *tmp_version;
+
+  /* This hangs on some systems (?)
+  system ("touch conf.sdltest");
+  */
+  { FILE *fp = fopen("conf.sdltest", "a"); if ( fp ) fclose(fp); }
+
+  /* HP/UX 9 (%@#!) writes to sscanf strings */
+  tmp_version = my_strdup("$min_sdl_version");
+  if (sscanf(tmp_version, "%d.%d.%d", &major, &minor, &micro) != 3) {
+     printf("%s, bad version string\n", "$min_sdl_version");
+     exit(1);
+   }
+
+   if (($sdl_major_version > major) ||
+      (($sdl_major_version == major) && ($sdl_minor_version > minor)) ||
+      (($sdl_major_version == major) && ($sdl_minor_version == minor) && ($sdl_micro_version >= micro)))
+    {
+      return 0;
+    }
+  else
+    {
+      printf("\n*** 'sdl2-config --version' returned %d.%d.%d, but the minimum version\n", $sdl_major_version, $sdl_minor_version, $sdl_micro_version);
+      printf("*** of SDL required is %d.%d.%d. If sdl2-config is correct, then it is\n", major, minor, micro);
+      printf("*** best to upgrade to the required version.\n");
+      printf("*** If sdl2-config was wrong, set the environment variable SDL2_CONFIG\n");
+      printf("*** to point to the correct copy of sdl2-config, and remove the file\n");
+      printf("*** config.cache before re-running configure\n");
+      return 1;
+    }
+}
+
+],, no_sdl=yes,[echo $ac_n "cross compiling; assumed OK... $ac_c"])
+        CFLAGS="$ac_save_CFLAGS"
+        CXXFLAGS="$ac_save_CXXFLAGS"
+        LIBS="$ac_save_LIBS"
+
+      fi
+      if test "x$sdl_pc" = "xno"; then
+        if test "x$no_sdl" = "xyes"; then
+          AC_MSG_RESULT(no)
+        else
+          AC_MSG_RESULT(yes)
+        fi
+      fi
+    fi
+  fi
+  if test "x$no_sdl" = x ; then
+     ifelse([$2], , :, [$2])
+  else
+     if test "$SDL2_CONFIG" = "no" ; then
+       echo "*** The sdl2-config script installed by SDL could not be found"
+       echo "*** If SDL was installed in PREFIX, make sure PREFIX/bin is in"
+       echo "*** your path, or set the SDL2_CONFIG environment variable to the"
+       echo "*** full path to sdl2-config."
+     else
+       if test -f conf.sdltest ; then
+        :
+       else
+          echo "*** Could not run SDL test program, checking why..."
+          CFLAGS="$CFLAGS $SDL_CFLAGS"
+          CXXFLAGS="$CXXFLAGS $SDL_CFLAGS"
+          LIBS="$LIBS $SDL_LIBS"
+          AC_TRY_LINK([
+#include <stdio.h>
+#include "SDL.h"
+
+int main(int argc, char *argv[])
+{ return 0; }
+#undef  main
+#define main K_and_R_C_main
+],      [ return 0; ],
+        [ echo "*** The test program compiled, but did not run. This usually means"
+          echo "*** that the run-time linker is not finding SDL or finding the wrong"
+          echo "*** version of SDL. If it is not finding SDL, you'll need to set your"
+          echo "*** LD_LIBRARY_PATH environment variable, or edit /etc/ld.so.conf to point"
+          echo "*** to the installed location  Also, make sure you have run ldconfig if that"
+          echo "*** is required on your system"
+	  echo "***"
+          echo "*** If you have an old version installed, it is best to remove it, although"
+          echo "*** you may also be able to get things to work by modifying LD_LIBRARY_PATH"],
+        [ echo "*** The test program failed to compile or link. See the file config.log for the"
+          echo "*** exact error that occured. This usually means SDL was incorrectly installed"
+          echo "*** or that you have moved SDL since it was installed. In the latter case, you"
+          echo "*** may want to edit the sdl2-config script: $SDL2_CONFIG" ])
+          CFLAGS="$ac_save_CFLAGS"
+          CXXFLAGS="$ac_save_CXXFLAGS"
+          LIBS="$ac_save_LIBS"
+       fi
+     fi
+     SDL_CFLAGS=""
+     SDL_LIBS=""
+     ifelse([$3], , :, [$3])
+  fi
+  AC_SUBST(SDL_CFLAGS)
+  AC_SUBST(SDL_LIBS)
+  rm -f conf.sdltest
+])
+
+dnl Check for SDL
+SDL_VERSION=2.0
+AM_PATH_SDL2($SDL_VERSION, sdl_pkgconfig=yes, sdl_pkgconfig=no)
+
+AM_CONDITIONAL([HAVE_SDL], [test x$sdl_pkgconfig = xyes])
+
 # Define localedir
 AC_DEFUN([V4L_EXPAND_PREFIX], [
 	$1=$2
@@ -507,6 +748,7 @@ compile time options summary
     pthread                    : $have_pthread
     QT version                 : $QT_VERSION
     ALSA support               : $USE_ALSA
+    SDL support		       : $sdl_pkgconfig
 
     build dynamic libs         : $enable_shared
     build static libs          : $enable_static
diff --git a/contrib/test/Makefile.am b/contrib/test/Makefile.am
index 4641e21..4e585cf 100644
--- a/contrib/test/Makefile.am
+++ b/contrib/test/Makefile.am
@@ -16,6 +16,10 @@ if HAVE_GLU
 noinst_PROGRAMS += v4l2gl
 endif
 
+if HAVE_SDL
+noinst_PROGRAMS += sdlcam
+endif
+
 driver_test_SOURCES = driver-test.c
 driver_test_LDADD = ../../utils/libv4l2util/libv4l2util.la
 
@@ -31,6 +35,11 @@ v4l2gl_SOURCES = v4l2gl.c
 v4l2gl_LDFLAGS = $(X11_LIBS) $(GL_LIBS) $(GLU_LIBS) $(ARGP_LIBS)
 v4l2gl_LDADD = ../../lib/libv4l2/libv4l2.la ../../lib/libv4lconvert/libv4lconvert.la
 
+sdlcam_LDFLAGS = $(JPEG_LIBS) $(SDL_LIBS) ../../lib/libv4l2/.libs/libv4l2.a  ../../lib/libv4lconvert/.libs/libv4lconvert.a
+sdlcam_CFLAGS = -I../..
+v4l2gl_LDADD = 
+# ../../lib/libv4l2/libv4l2.la ../../lib/libv4lconvert/libv4lconvert.la
+
 mc_nextgen_test_CFLAGS = $(LIBUDEV_CFLAGS)
 mc_nextgen_test_LDFLAGS = $(LIBUDEV_LIBS)
 
diff --git a/contrib/test/libcam.c b/contrib/test/libcam.c
new file mode 100644
index 0000000..115e6b1
--- /dev/null
+++ b/contrib/test/libcam.c
@@ -0,0 +1,48 @@
+static void fmt_print(struct v4l2_format *fmt)
+{
+	int f;
+	printf("Format: %dx%d. ", fmt->fmt.pix.width, fmt->fmt.pix.height);
+	printf("%x ", fmt->fmt.pix.pixelformat);
+	f = fmt->fmt.pix.pixelformat;
+	for (int i=0; i<4; i++) {
+		printf("%c", f & 0xff);
+		f >>= 8;
+	}
+	printf("\n");
+}
+
+static void pgm_write(struct v4l2_format *fmt, unsigned char *img, char *out_name, char *extra)
+{
+	FILE *fout;
+	int size = fmt->fmt.pix.width * fmt->fmt.pix.height;
+	char swapped[size*2];
+
+	fout = fopen(out_name, "w");
+	if (!fout) {
+		perror("Cannot open image");
+		exit(EXIT_FAILURE);
+	}
+	switch (fmt->fmt.pix.pixelformat) {
+		/* ?? 	cinfo.in_color_space = JCS_YCbCr; */
+	case V4L2_PIX_FMT_SGRBG10:
+		printf("ok\n");
+		break;
+	default:
+		printf("Bad pixel format\n");
+		exit(1);
+	}
+
+	for (int i=0; i<size*2; i+=2) {
+		swapped[i] = img[i+1];
+		swapped[i+1] = img[i];
+	}
+
+	fprintf(fout, "P5\n");
+	if (extra)
+		fprintf(fout, "%s", extra);
+	  
+	fprintf(fout, "%d %d\n1023\n",
+		fmt->fmt.pix.width, fmt->fmt.pix.height);
+	fwrite(swapped, size, 2, fout);
+	fclose(fout);
+}
diff --git a/contrib/test/raw.c b/contrib/test/raw.c
new file mode 100644
index 0000000..70d3340
--- /dev/null
+++ b/contrib/test/raw.c
@@ -0,0 +1,298 @@
+/* -*- c-file-style: "linux" -*- */
+/*
+   lens-shading is described at page 1411 of omap3isp manual,
+   downsampled by 4..64
+
+   gcc -std=c99 raw.c
+*/
+
+#include <linux/videodev2.h>
+#include <arpa/inet.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+#include "libcam.c"
+
+struct image {
+	struct v4l2_format fmt;
+	unsigned char *buf;
+	unsigned char data[]; 
+};
+
+struct image *read_pgm(char *name)
+{
+	FILE *f;
+	char buf[1024];
+	struct v4l2_format fmt;
+	int max;
+
+
+	f = fopen(name, "r");
+	if (!f) {
+		printf("Can not open: %s\n", name);
+		return NULL;
+	}
+	fgets(buf, sizeof(buf)-1, f);
+	if (strcmp(buf, "P5\n")) {
+		printf("Not a pgm\n");
+		return NULL;
+	}
+
+	fgets(buf, sizeof(buf)-1, f);
+	while (buf[0] == '#')
+		fgets(buf, sizeof(buf)-1, f);
+
+	fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	fmt.fmt.pix.field = V4L2_FIELD_NONE;
+
+	if (2 != sscanf(buf, "%d %d\n", &fmt.fmt.pix.width, &fmt.fmt.pix.height))
+		printf("Could not read size: %s\n", buf);
+	fgets(buf, sizeof(buf)-1, f);
+	if (1 != sscanf(buf, "%d\n", &max))
+		printf("Could not read bits\n");
+
+	int size = fmt.fmt.pix.width * fmt.fmt.pix.height;
+	int bpp;
+	switch (max) {
+	case 255:
+		bpp = 1;
+		fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG8;		
+		break;
+	case 1023:
+		bpp = 2;
+		fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG10;
+		break;
+	default:
+		printf("Unexpected image depth: %d\n", max);
+		return NULL;
+	}
+
+	struct image *img = malloc(sizeof(struct image) + size*bpp);
+	img->buf = img->data;
+	fmt_print(&fmt);
+
+	if (size != fread(img->buf, bpp, size, f))
+		printf("Short read, bpp %d\n", bpp);
+
+	if (bpp == 2)
+		for (int i = 0; i < size; i++) {
+			unsigned short *b = (void *) img->buf;
+			b[i] = ntohs(b[i]);
+		}
+
+	img->fmt = fmt;
+
+	fclose(f);
+	return img;
+}
+
+static inline int img_pos(struct image *img, int x, int y)
+{
+	return y*img->fmt.fmt.pix.width + x;
+}
+
+static inline unsigned short img_value(struct image *img, int x, int y)
+{
+	int pos = img_pos(img, x, y);
+
+	switch (img->fmt.fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG8:
+		return img->buf[pos];
+	case  V4L2_PIX_FMT_SGRBG10:
+		return ((unsigned short *)img->buf)[pos];
+	default:
+		printf("Unknown format in img_value\n");
+		exit(1);
+	}
+}
+
+static int img_value_safe(struct image *img, int x, int y)
+{
+	if ((x < 0) || (y < 0))
+		return -1;
+	if ((x >= img->fmt.fmt.pix.width) || (y >= img->fmt.fmt.pix.height))
+		return -1;
+	return img_value(img, x, y);
+}
+
+static inline void img_value_set(struct image *img, int x, int y, int v)
+{
+	int pos = img_pos(img, x, y);
+
+	switch (img->fmt.fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG8:
+		img->buf[pos] = v;
+		break;
+	case  V4L2_PIX_FMT_SGRBG10:
+		((unsigned short *)img->buf)[pos] = v;
+		break;
+	default:
+		printf("Unknown format in img_value_set\n");
+		exit(1);
+	}
+}
+
+static inline int img_max_value(struct image *img)
+{
+	switch (img->fmt.fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG8:
+		return 255;
+	case  V4L2_PIX_FMT_SGRBG10:
+		return 1023;
+	default:
+		printf("Unknown format in img_value_max\n");
+		exit(1);
+	}
+}
+
+static int img_expected_value(struct image *img, int x, int y)
+{
+	int v = 0, n = 0, v1;
+
+	v1 = img_value_safe(img, x-2, y);
+	if (v1 >= 0)
+		v += v1, n++;
+
+	v1 = img_value_safe(img, x+2, y);
+	if (v1 >= 0)
+		v += v1, n++;
+
+	v1 = img_value_safe(img, x, y-2);
+	if (v1 >= 0)
+		v += v1, n++;
+
+	v1 = img_value_safe(img, x, y+2);
+	if (v1 >= 0)
+		v += v1, n++;
+
+	v /= n;
+	return v;
+}
+
+/* img -= sub */
+static void img_substract(struct image *img, struct image *sub)
+{
+	for (int y=0; y<img->fmt.fmt.pix.height; y++)
+		for (int x=0; x<img->fmt.fmt.pix.width; x++) {
+			int v = img_value(img, x, y) -
+				img_value(sub, x, y);
+			if (v < 0)
+				v = 0;
+			img_value_set(img, x, y, v);
+		}
+}
+
+static int histogram_perc(int *cdf, int buckets, int perc)
+{
+	long long total = cdf[buckets-1];	
+	int lim = total * perc / 10000;
+
+	for (int i=0; i<buckets; i++)
+		if (cdf[i] > lim)
+			return i;
+	return buckets-1;
+}
+
+static void img_histogram(struct image *img, int cdf[], int buckets, int step)
+{
+	int max = img_max_value(img);
+
+	for (int i=0; i<buckets; i++)
+		cdf[i] = 0;
+	
+	for (int y = 0; y < img->fmt.fmt.pix.height; y+=step)
+		for (int x = 0; x < img->fmt.fmt.pix.width; x+=step) {
+			int b;
+			b = img_value(img, x, y);
+			b = (b * buckets)/(max+1);
+			cdf[b]++;
+		}
+
+	for (int i=1; i<buckets; i++)
+		cdf[i] += cdf[i-1];
+}
+
+static void img_remove_dead(struct image *img, struct image *sub)
+{
+	int buckets = 1024;
+	int cdf[buckets];
+	int lim;
+	img_histogram(sub, cdf, buckets, 1);
+
+	printf("Total is %d\n", cdf[buckets-1]);
+	printf("Median is %d\n", histogram_perc(cdf, buckets, 5000));
+	lim = histogram_perc(cdf, buckets, 9999);
+	printf("99th percentile is %d (%d)\n", lim, lim/4);
+	lim = lim + lim / 10;
+	printf("bad pixel limit is %d (%d)\n", lim, lim/4);
+	printf("%d bad pixels\n", cdf[buckets-1] - cdf[lim]);
+
+	int c = 0;
+
+	for (int y = 0; y < img->fmt.fmt.pix.height; y++)
+		for (int x = 0; x < img->fmt.fmt.pix.width; x++) {
+			int v = img_value(sub, x, y);
+			if (v > lim) {
+				int vo = img_value(img, x, y);
+				int v1 = img_expected_value(img, x, y);
+
+				img_value_set(img, x, y, v1);
+				//printf("Bad pixel, dark: %d %d -> %d\n", v, vo, v1);
+				c++;
+			}
+		}
+	printf("Corrected %d bad pixels\n", c);
+
+}
+
+static void img_shading(struct image *img, struct image *cor)
+{
+	int syi = img->fmt.fmt.pix.height;
+	int syc = cor->fmt.fmt.pix.height;
+	int sxi = img->fmt.fmt.pix.width;
+	int sxc = cor->fmt.fmt.pix.width;
+	int max = 0, min = 255;
+	
+	for (int y=0; y<syc; y++)
+		for (int x=0; x<sxc; x++) {
+			int v = img_value(cor, x, y);
+
+			if (v < min)
+				min = v;
+			if (v > max)
+				max = v;
+		}
+
+	printf("Extremes in correction image are %d, %d\n", min, max);
+
+	int m = img_max_value(img);
+	for (int y=0; y<syi; y++) {
+		int yc = (y*syc) / syi;
+		for (int x=0; x<sxi; x++) {
+			int xc = (x*sxc) / sxi;
+			int v = img_value(img, x, y);
+			int vc = img_value(cor, xc, yc);
+
+			v = (v * max) / vc;
+			if (v > m)
+				v = m;
+			img_value_set(img, x, y, v);
+
+		}
+	}
+}
+
+int main(void)
+{
+	struct image *img = read_pgm("/data/tmp/test_dark.pgm");
+	struct image *dark = read_pgm("/data/tmp/dark_max_iso_exp.pgm");
+	struct image *shading = read_pgm("/data/tmp/lens_shading.pgm");
+
+	pgm_write(&img->fmt, img->buf, "/data/tmp/delme_original.pgm", NULL);
+	//img_substract(img, dark);
+	img_remove_dead(img, dark);
+	pgm_write(&img->fmt, img->buf, "/data/tmp/delme_dark.pgm", NULL);
+	img_shading(img, shading);
+	pgm_write(&img->fmt, img->buf, "/data/tmp/delme_shading.pgm", NULL);
+}
diff --git a/contrib/test/sdlcam.c b/contrib/test/sdlcam.c
new file mode 100644
index 0000000..69b9712
--- /dev/null
+++ b/contrib/test/sdlcam.c
@@ -0,0 +1,1385 @@
+/* -*- c-file-style: "linux" -*- */
+/* Digital still camera.
+
+   SDL based, suitable for camera phone such as Nokia N900. In
+   particular, we support focus, gain and exposure control, but not
+   aperture control or lens zoom.
+
+   Copyright 2017 Pavel Machek, LGPL
+
+*/
+
+#include <time.h>
+#include <errno.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+#include <sys/types.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/ioctl.h>
+#include <fcntl.h>
+
+#include <jpeglib.h>
+
+#include "libv4l2.h"
+#include <linux/videodev2.h>
+#define COOKED
+#ifdef COOKED
+#include "../../lib/libv4l2/libv4l2-priv.h"
+#endif
+#include "libv4l-plugin.h"
+
+#include <SDL2/SDL.h>
+#include <SDL2/SDL_image.h>
+
+#include "libcam.c"
+
+static double dtime(void)
+{
+	static double start = 0.0;
+	struct timeval now;
+
+	gettimeofday(&now, NULL);
+
+	double n = now.tv_sec + now.tv_usec / 1000000.;
+	if (!start)
+		start = n;
+	return n - start;
+}
+
+static long v4l2_g_ctrl(int fd, long id)
+{
+	int res;
+	struct v4l2_control ctrl;
+	ctrl.id = id;
+	res = v4l2_ioctl(fd, VIDIOC_G_CTRL, &ctrl);
+	if (res < 0)
+		printf("Get control %ld failed\n", id);
+	return ctrl.value;
+}
+
+static int v4l2_s_ctrl(int fd, long id, long value)
+{
+	int res;
+	struct v4l2_control ctrl;
+	ctrl.id = id;
+	ctrl.value = value;
+	res = v4l2_ioctl(fd, VIDIOC_S_CTRL, &ctrl);
+	if (res < 0)
+		printf("Set control %lx %ld failed\n", id, value);
+	return res;
+}
+
+
+static int v4l2_set_focus(int fd, int diopt)
+{
+	if (v4l2_s_ctrl(fd, V4L2_CID_FOCUS_ABSOLUTE, diopt) < 0) {
+		printf("Could not set focus\n");
+	}
+	return 0;
+}
+
+#define SX 2592
+#define SY 1968
+#define SIZE SX*SY*3
+struct dev_info {
+	int fd;
+	struct v4l2_format fmt;
+
+	unsigned char buf[SIZE];
+	int debug;
+#define D_TIMING 1
+};
+
+struct sdl {
+	SDL_Window *window;
+	SDL_Surface *screen, *liveview;
+
+	int wx, wy; /* Window size */
+	int sx, sy; /* Live view size */
+	int bx, by; /* Border size */
+	int nx, ny; /* Number of buttons */
+	float factor;
+
+	/* These should go separately */
+	int do_focus, do_exposure, do_flash, do_white, do_big, do_full;
+	double zoom;
+	double focus_min;
+
+	int slider_mode;
+#define M_BIAS 0
+#define M_EXPOSURE 1
+#define M_GAIN 2
+#define M_FOCUS 3
+#define M_NUM 4
+	
+	int fd;
+
+	struct dev_info *dev;
+};
+
+typedef struct {
+	uint8_t r, g, b, alpha;
+} pixel;
+
+#define d_raw 1
+
+static void sfc_put_pixel(SDL_Surface* liveview, int x, int y, pixel *p)
+{
+	Uint32* p_liveview = (Uint32*)liveview->pixels;
+	p_liveview += y*liveview->w+x;
+	*p_liveview = SDL_MapRGBA(liveview->format, p->r, p->g, p->b, p->alpha);
+}
+
+static void sdl_begin_paint(struct sdl *m)
+{
+	/* Fill the surface white */
+	SDL_FillRect(m->liveview, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+
+	SDL_LockSurface(m->liveview);
+}
+
+static void sdl_finish_paint(struct sdl *m) {
+	SDL_UnlockSurface(m->liveview);
+	SDL_Rect rcDest = { m->bx, m->by, m->sx, m->sy };
+
+	SDL_BlitSurface(m->liveview, NULL, m->screen, &rcDest);
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+  
+static void sdl_paint_image(struct sdl *m, char **xpm, int x, int y) {
+	SDL_Surface *image = IMG_ReadXPMFromArray(xpm);
+	if (!image) {
+		printf("IMG_Load: %s\n", IMG_GetError());
+		exit(1);
+	}
+
+	int x_pos = x - image->w/2, y_pos = y - image->h/2;
+
+	SDL_Rect rcDest = { x_pos, y_pos, image->w, image->h };
+	int r = SDL_BlitSurface(image, NULL, m->screen, &rcDest);
+
+	if (r) {
+		printf("Error blitting: %s\n", SDL_GetError());
+		exit(1);
+	}
+	SDL_FreeSurface(image);
+}
+
+static void cam_exposure_limits(struct sdl *m, struct v4l2_queryctrl *qctrl)
+{
+	qctrl->id = V4L2_CID_EXPOSURE_ABSOLUTE;
+	
+	if (v4l2_ioctl(m->fd, VIDIOC_QUERYCTRL, qctrl)) {
+		printf("Exposure absolute limits failed\n");
+		exit(1);
+	}
+
+	/* Minimum of 11300 gets approximately same range on ISO and
+	 * exposure axis. */
+	if (qctrl->minimum < 500)
+		qctrl->minimum = 500;
+}
+
+static void cam_set_exposure(struct sdl *m, double v)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	double res;
+	double range;
+	struct v4l2_queryctrl qctrl = { .id = cid };
+	struct v4l2_control ctrl = { .id = cid };
+
+	cam_exposure_limits(m, &qctrl);
+
+	if (v4l2_ioctl(m->fd, VIDIOC_G_CTRL, &ctrl)) {
+		printf("Can't get exposure parameters\n");
+		exit(1);
+	}
+
+	range = log2(qctrl.maximum) - log2(qctrl.minimum);
+	res = log2(qctrl.minimum) + v*range;
+	res = exp2(res);
+
+	v4l2_s_ctrl(m->fd, V4L2_CID_EXPOSURE_ABSOLUTE, res);
+}
+
+static double cam_convert_exposure(struct sdl *m, int v)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	double res;
+	struct v4l2_queryctrl qctrl = { .id = cid };
+
+	cam_exposure_limits(m, &qctrl);
+	res = (log2(v) - log2(qctrl.minimum)) / (log2(qctrl.maximum) - log2(qctrl.minimum));
+
+	return res;
+}
+
+static double cam_get_exposure(struct sdl *m)
+{
+	int cid = V4L2_CID_EXPOSURE_ABSOLUTE;
+	struct v4l2_control ctrl = { .id = cid };
+
+	if (v4l2_ioctl(m->fd, VIDIOC_G_CTRL, &ctrl))
+		return -1;
+
+	return cam_convert_exposure(m, ctrl.value);
+}
+
+static int cam_get_gain_iso(struct dev_info *dev)
+{
+	double x;
+
+	x = v4l2_g_ctrl(dev->fd, 0x00980913);
+	x = (x / 10); /* EV */
+	x = exp2(x);
+	x = 100*x;
+	return x;
+}
+
+static void cam_set_focus(struct sdl *m, double val)
+{
+	v4l2_s_ctrl(m->fd, V4L2_CID_FOCUS_ABSOLUTE, (val * m->focus_min) * (1023. / 20));
+}
+
+static double cam_convert_focus(struct sdl *m, double diopter)
+{
+	return diopter / m->focus_min;
+}
+
+static double cam_get_focus_diopter(struct sdl *m)
+{
+	return (v4l2_g_ctrl(m->fd, V4L2_CID_FOCUS_ABSOLUTE) * 20.) / 1023.;
+}
+
+static double cam_get_focus(struct sdl *m)
+{
+	return cam_convert_focus(m, cam_get_focus_diopter(m));
+}
+
+static void sdl_line_color(struct sdl *m, int x1, int y1, int x2, int y2, Uint32 color)
+{
+	SDL_Rect rcDest = { x1, y1, x2-x1+1, y2-y1+1};
+
+	SDL_FillRect(m->screen, &rcDest, color);
+}
+
+static void sdl_line(struct sdl *m, int x1, int y1, int x2, int y2)
+{
+	sdl_line_color(m, x1, y1, x2, y2, SDL_MapRGB( m->liveview->format, 255, 255, 255 ));
+}
+
+static void sdl_digit(struct sdl *m, int x, int y, int s, int i)
+{
+	unsigned char gr[] = { 0x5f, 0x0a, 0x76, 0x7a, 0x2b,
+			       0x79, 0x7d, 0x1a, 0x7f, 0x7b };
+	unsigned char g = gr[i];
+	/*
+              10
+	    01  02
+              20
+            04  08
+	      40
+	*/
+
+	if (g & 1) sdl_line(m, x, y, x, y+s);
+	if (g & 2) sdl_line(m, x+s, y, x+s, y+s);
+	if (g & 4) sdl_line(m, x, y+s, x, y+s+s);
+	if (g & 8) sdl_line(m, x+s, y+s, x+s, y+s+s);
+
+	if (g & 0x10) sdl_line(m, x, y, x+s, y);
+	if (g & 0x20) sdl_line(m, x, y+s, x+s, y+s);
+	if (g & 0x40) sdl_line(m, x, y+s+s, x+s, y+s+s);
+}
+
+static void sdl_number(struct sdl *m, int x, int y, int s, int digits, int i)
+{
+	int tot = s * 5;
+	x += tot * digits;
+	for (int j=0; j<digits; j++) {
+		sdl_digit(m, x+2, y+4, s*3, i%10);
+		i /= 10;
+		x -= tot;
+	}
+}
+
+static void sdl_icon_pos(struct sdl *m, int *x, int *y, double val)
+{
+	*x = m->wx - 10;
+	*y = val * m->wy;
+} 
+
+static void sdl_paint_ui_iso(struct sdl *m, double y_, int i)
+{
+	static char *iso_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		".x..xx..x.......",
+		".x.x...x.x......",
+		".x..x..x.x......",
+		".x...x.x.x......",
+		".x.xx...x.......",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+
+	int x, y;
+
+	sdl_icon_pos(m, &x, &y, y_);
+	sdl_number(m, x-35, y-10, 1, 3, i);
+	sdl_paint_image(m, iso_xpm, x, y);
+}
+
+static void sdl_paint_ui_exposure(struct sdl *m, int t)
+{
+	static char *time_1_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"......x.........",
+		".....x..........",
+		"....x...........",
+		"...x............",
+		"................",
+		".xxx.xxxx.xxxx..",
+		"x....x....x.....",
+		".xx..xxx..x.....",
+		"...x.x....x.....",
+		"...x.x....x.....",
+		"xxx..xxxx.xxxx..",
+		"................",
+	};
+	int x, y;
+
+	sdl_icon_pos(m, &x, &y, cam_convert_exposure(m, 1000000/t));
+	sdl_number(m, x-35, y-10, 1, 3, t);
+	sdl_paint_image(m, time_1_xpm, x, y);
+}
+
+static void sdl_paint_boolean(struct sdl *m, char **image, int x, int y, int yes)
+{
+	static char *not_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"......xxxxx.....",
+		"....xx.....xx...",
+		"...x.........x..",
+		"..x........xx.x.",
+		"..x......xx...x.",
+		".x.....xx......x",
+		".x...xx........x",
+		"..xxx.........x.",
+		"..x...........x.",
+		"...x.........x..",
+		"....xx.....xx...",
+		"......xxxxx.....",
+	};
+
+	sdl_paint_image(m, image, x, y);
+	if (!yes)
+		sdl_paint_image(m, not_xpm,  16+x, y);      
+}
+
+static void sdl_paint_button(struct sdl *m, char **image, int x, int y, int yes)
+{
+	int bsx = m->wx/m->nx;
+	int bsy = m->wy/m->ny;
+	if (x < 0)
+		x += m->nx;
+	if (y < 0)
+		y += m->ny;
+	x = bsx/2 + x*bsx;
+	y = bsy/2 + y*bsy;
+	sdl_paint_boolean(m, image, x, y, yes);
+}
+
+static void sdl_paint_ui_focus(struct sdl *m, int f)
+{
+	static char *cm_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"....###..#.#.##.",
+		"...#.....##.#..#",
+		"...#.....#..#..#",
+		"...#.....#..#..#",
+		"....###..#..#..#",
+		"................",
+	};
+	double dioptr = 1/(f/100.);
+	int x, y;
+
+	if (dioptr > m->focus_min)
+		return;
+
+	sdl_icon_pos(m, &x, &y, cam_convert_focus(m, dioptr));
+	sdl_paint_image(m, cm_xpm, x, y);
+	sdl_number(m, x-12, y-15, 1, 3, f);
+}
+
+static void sdl_paint_ui_bias(struct sdl *m, double v)
+{
+	static char *bias_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"...#............",
+		"...#.......#....",
+		".#####....#.....",
+		"...#.....#......",
+		"...#....#.......",
+		".......#........",
+		"......#...#####.",
+		".....#..........",
+		"....#...........",
+		"...#............",
+		"................",
+		"................",
+	};
+	int x, y;
+
+	sdl_icon_pos(m, &x, &y, v);
+	sdl_paint_image(m, bias_xpm, x, y);
+	//sdl_number(m, x-12, y-15, 1, 3, f);
+}
+
+static void sdl_paint_slider(struct sdl *m)
+{
+	switch (m->slider_mode) {
+	case M_BIAS:
+		sdl_paint_ui_bias(m, 0.5);
+		return;
+	case M_EXPOSURE:
+		sdl_paint_ui_exposure(m, 10);
+		sdl_paint_ui_exposure(m, 100);
+		sdl_paint_ui_exposure(m, 999);
+#if 0
+		sdl_paint_ui_exposure(m, 10001);
+		sdl_paint_ui_exposure(m, 14002);
+#endif
+		return;
+	case M_GAIN:
+		sdl_paint_ui_iso(m, 0/4., 100);
+		sdl_paint_ui_iso(m, 1/4., 200);
+		sdl_paint_ui_iso(m, 2/4., 400);
+		sdl_paint_ui_iso(m, 3/4., 800);
+		return;
+	case M_FOCUS:
+		sdl_paint_ui_focus(m, 100);
+		sdl_paint_ui_focus(m, 40);
+		sdl_paint_ui_focus(m, 25);
+		sdl_paint_ui_focus(m, 16);	
+		sdl_paint_ui_focus(m, 10);
+		sdl_paint_ui_focus(m, 8);
+		sdl_paint_ui_focus(m, 6);
+		return;
+	}
+}
+
+static void sdl_paint_ui(struct sdl *m)
+{
+	static char *wait_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"....########....",
+		".....#....#.....",
+		".....#....#.....",
+		"......#..#......",
+		".......##.......",
+		"......#..#......",
+		".....#....#.....",
+		".....#....#.....",
+		"....########....",
+	};
+
+	static char *ok_xpm[] = {
+		"16 9 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"...............#",
+		"............###.",
+		"..........##....",
+		"#.......##......",
+		".#.....#........",
+		"..#...#.........",
+		"..#..#..........",
+		"...##...........",
+		"...#............",
+	};
+
+	static char *exit_xpm[] = {
+		"16 9 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"....x......x....",
+		".....x....x.....",
+		"......x..x......",
+		".......xx.......",
+		".......xx.......",
+		"......x..x......",
+		".....x....x.....",
+		"....x......x....",
+		"................",
+	};
+
+	static char *af_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..x..........",
+		"................",
+		"................",
+	};
+
+	static char *ae_xpm[] = {
+		"16 12 2 1",
+		"x c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		".....xxxxxxx....",
+		".....x..........",
+		".....x..........",
+		".x...xxxxx......",
+		"x.x..x..........",
+		"xxx..x..........",
+		"x.x..x..........",
+		"x.x..xxxxxxx....",
+		"................",
+		"................",
+	};
+    
+	static char *focus_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"###..........###",
+		"#..............#",
+		"#.....####.....#",
+		".....#....#.....",
+		".....#....#.....",
+		"#.....####.....#",
+		"#..............#",
+		"###..........###",
+		"................",
+		"................",
+	};
+
+	static char *flash_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",		
+		"..........#.....",
+		"........##......",
+		".......##.......",
+		"......##........",
+		".....########...",
+		"..........##....",
+		".......#.##.....",
+		".......###......",
+		".......####.....",
+		"................",
+		"................",
+	};
+
+	static char *wb_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"#.....#..####...",
+		"#.....#..#...#..",
+		"#..#..#..####...",
+		"#..#..#..#...#..",
+		".##.##...####...",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+/* Template for more xpm's:
+	static char *empty_xpm[] = {
+		"16 12 2 1",
+		"# c #ffffff",
+		". c #000000",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+		"................",
+	};
+*/
+	SDL_FillRect(m->screen, NULL, SDL_MapRGB( m->liveview->format, 0, 0, 0 ));
+
+	{
+		/* Paint grid */
+		int x, y;
+		int nx = m->nx;
+		for (x=1; x<nx; x++) {
+			int x_ = (x*m->wx)/nx;
+			sdl_line_color(m, x_, 1, x_, m->wy-1, SDL_MapRGB( m->liveview->format, 40, 40, 40 ));
+		}
+
+		int ny = m->ny;
+		for (y=1; y<nx; y++) {
+			int y_ = (y*m->wy)/ny;
+			sdl_line_color(m, 1, y_, m->wx-1, y_, SDL_MapRGB( m->liveview->format, 40, 40, 40 ));
+		}
+				       
+	}
+
+	sdl_paint_image(m, wait_xpm,  m->wx/2,     m->wy/2);
+
+
+	sdl_paint_button(m, af_xpm, 0,  0, m->do_focus);
+	sdl_paint_button(m, ae_xpm, -1, 0, m->do_exposure);
+
+	sdl_paint_button(m, exit_xpm,   0, -1, 1);
+	sdl_paint_button(m, flash_xpm,  1, -1, m->do_flash);
+	sdl_paint_button(m, wb_xpm,     2, -1, m->do_white);
+	sdl_paint_button(m, focus_xpm, -1, -2, 1);
+	sdl_paint_button(m, ok_xpm,    -1, -1, 1);
+
+	sdl_paint_slider(m);
+	SDL_UpdateWindowSurface(m->window);
+}
+
+static double usec_to_time(double v)
+{
+	return 1/(v*.000001);
+}
+
+static void sdl_status_below(struct sdl *m, double avg)
+{
+	int ox = m->bx;
+	int oy = m->by+m->sy;
+	SDL_Rect rcDest = { ox, oy, m->sx, 25 /* m->by */ };
+
+	SDL_FillRect(m->screen, &rcDest, SDL_MapRGB( m->liveview->format, 30, 30, 30 ));
+
+	ox+=40;
+	sdl_number(m, ox, oy, 2, 3, avg*1000);
+
+	{
+		double focus, gain, exposure;
+
+		exposure = v4l2_g_ctrl(m->fd, V4L2_CID_EXPOSURE_ABSOLUTE);
+		gain = v4l2_g_ctrl(m->fd, 0x00980913);
+		focus = cam_get_focus_diopter(m);
+
+		ox+=40;
+		double x = usec_to_time(exposure);
+		if (x > 999) x = 999;
+		sdl_number(m, ox, oy, 2, 3, x);
+
+		ox+=40;
+		x = (gain / 10);
+		sdl_number(m, ox, oy, 2, 1, x);
+
+		ox+=20;
+		x = focus; /* diopters */
+		if (x == 0)
+			x = 999;
+		else
+			x = 100/x; /* centimeters */
+		sdl_number(m, ox, oy, 2, 3, x);
+	}
+
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+
+static void sdl_status(struct sdl *m, double avg)
+{
+	int ox = 0;
+	int oy = m->wy/2;
+	int s = 3;
+	SDL_Rect rcDest = { ox, oy, s*25, s*40 };
+
+	SDL_FillRect(m->screen, &rcDest, SDL_MapRGB( m->liveview->format, 30, 30, 30 ));
+
+	sdl_number(m, ox, oy, s, 3, avg*1000);
+	oy += s*10;
+
+	{
+		double focus, gain, exposure;
+
+		exposure = v4l2_g_ctrl(m->fd, V4L2_CID_EXPOSURE_ABSOLUTE);
+		gain = cam_get_gain_iso(m->dev);
+		focus = cam_get_focus_diopter(m);
+
+		double x = usec_to_time(exposure);
+		if (x > 999) x = 999;
+		sdl_number(m, ox, oy, s, 3, x);
+
+		oy += s*10;
+		if (gain > 999)
+			gain = 999;
+		sdl_number(m, ox, oy, s, 3, gain);
+
+		oy += s*10;
+		x = focus; /* diopters */
+		if (x == 0)
+			x = 999;
+		else
+			x = 100/x; /* centimeters */
+		sdl_number(m, ox, oy, s, 3, x);
+	}
+
+	SDL_UpdateWindowSurfaceRects(m->window, &rcDest, 1);
+}
+static pixel buf_pixel(struct v4l2_format *fmt, unsigned char *buf, int x, int y)
+{
+	pixel p = { 0, 0, 0, 0 };
+	int pos = x + y*fmt->fmt.pix.width;
+
+	p.alpha = 128;
+
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+#if 0
+		b = buf[pos*2];
+		b += buf[pos*2+1] << 8;
+		b /= 4;
+		if ((y % 2) == 0) {
+			switch (x % 2) {
+			case 0: p.g = b/2; break;
+			case 1: p.r = b; break;
+			}
+		} else {
+			switch (x % 2) {
+			case 0: p.b = b; break;
+			case 1: p.g = b/2; break;
+			}
+		}
+#else
+		{
+			short *b2 = (void *)buf;
+			x &= ~1;
+			y &= ~1;
+			p.g = b2[x + y*fmt->fmt.pix.width] /4;
+			p.r = b2[x + y*fmt->fmt.pix.width+1] /4;
+			p.b = b2[x + (y+1)*fmt->fmt.pix.width] /4;
+		}
+#endif
+		break;
+
+	case V4L2_PIX_FMT_RGB24:
+		pos *= 3;
+		p.r = buf[pos];
+		p.g = buf[pos+1];
+		p.b = buf[pos+2];
+		break;
+
+	default:
+		printf("Wrong pixel format!\n");
+		fmt_print(fmt);
+		exit(1);
+	}
+
+	return p;
+}
+
+static char *fmt_name(struct v4l2_format *fmt)
+{
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+		return "GRBG10";
+	case V4L2_PIX_FMT_RGB24:
+		return "RGB24";
+	default:
+		return "unknown";
+	}
+}
+
+static void sdl_handle_focus(struct sdl *m, float how)
+{
+	v4l2_set_control(m->fd, V4L2_CID_FOCUS_ABSOLUTE, 65535. * how);
+}
+
+static void sdl_key(struct sdl *m, int c)
+{
+	switch (c) {
+	case 27: exit(1); break;
+	case 'q': sdl_handle_focus(m, 0.); break;
+	case 'w': sdl_handle_focus(m, 1/6.); break;
+	case 'e': sdl_handle_focus(m, 1/3.); break;
+	case 'r': sdl_handle_focus(m, 1/2.); break;
+	case 't': sdl_handle_focus(m, 1/1); break;
+	case 'y': sdl_handle_focus(m, 1/.8); break;
+	case 'u': sdl_handle_focus(m, 1/.5); break;
+	case 'i': sdl_handle_focus(m, 1/.2); break;
+	case 'o': sdl_handle_focus(m, 1/.1); break;
+	case 'p': sdl_handle_focus(m, 1/.05); break;
+	case SDLK_SPACE: /* save_image(); */ printf("Should save jpeg.\n"); break;
+	default: printf("Unknown key %d / %c", c, c);
+	}
+}
+
+static int sdl_render_statistics(struct sdl *m)
+{
+	pixel white;
+	double focus, gain, exposure;
+
+	white.r = (Uint8)0xff;
+	white.g = (Uint8)0xff;
+	white.b = (Uint8)0xff;
+	white.alpha = (Uint8)128;
+
+	exposure = cam_get_exposure(m);
+	gain = v4l2_get_control(m->fd, 0x00980913) / 65535.;
+	focus = cam_get_focus(m);
+
+	for (int x=0; x<m->sx && x<m->sx*focus; x++)
+		sfc_put_pixel(m->liveview, x, 0, &white);
+
+	for (int y=0; y<m->sy && y<m->sy*gain; y++)
+		sfc_put_pixel(m->liveview, 0, y, &white);
+
+	for (int y=0; y<m->sy && y<m->sy*exposure; y++)
+		sfc_put_pixel(m->liveview, m->sx-1, y, &white);
+
+	for (int x=0; x<m->sx; x++)
+		sfc_put_pixel(m->liveview, x, m->sy-1, &white);
+
+	return 0;
+}
+
+static void sdl_render(struct sdl *m, unsigned char *buf, struct v4l2_format *fmt)
+{
+	float zoom = m->zoom;
+	if (!m->window) 
+		return;
+
+	sdl_begin_paint(m);
+	int x0 = (fmt->fmt.pix.width - m->sx*m->factor/zoom)/2;
+	int y0 = (fmt->fmt.pix.height - m->sy*m->factor/zoom)/2;
+
+	for (int y = 0; y < m->sy; y++)
+		for (int x = 0; x < m->sx; x++) {
+			int x1 = x0+x*m->factor/zoom;
+			int y1 = y0+y*m->factor/zoom;
+			pixel p = buf_pixel(fmt, buf, x1, y1);
+			p.alpha = 128;
+			sfc_put_pixel(m->liveview, x, y, &p);
+		}
+
+	sdl_render_statistics(m);
+	sdl_finish_paint(m);
+}
+
+static void sdl_sync_settings(struct sdl *m)
+{
+	printf("Autofocus: "); v4l2_s_ctrl(m->fd, V4L2_CID_FOCUS_AUTO, m->do_focus);
+	printf("Autogain: " ); v4l2_s_ctrl(m->fd, V4L2_CID_AUTOGAIN, m->do_exposure);
+	printf("Autowhite: "); v4l2_s_ctrl(m->fd, V4L2_CID_AUTO_WHITE_BALANCE, m->do_white);
+	v4l2_s_ctrl(m->fd, 0x009c0901, m->do_flash ? 2 : 0);
+}
+
+static void sdl_init_window(struct sdl *m)
+{
+	if (m->do_full) {
+		m->wx = 800;
+		m->wy = 480;
+	} else {
+		m->wx = 800;
+		m->wy = 429;
+	}
+
+	SDL_SetWindowFullscreen(m->window, m->do_full*SDL_WINDOW_FULLSCREEN_DESKTOP);
+
+	m->screen = SDL_GetWindowSurface(m->window);
+	if (!m->screen) {
+		printf("Couldn't create screen\n");
+		exit(1);
+	}
+}
+
+static void sdl_init(struct sdl *m, struct dev_info *dev)
+{
+	m->fd = dev->fd;
+	m->dev = dev;
+
+	if (SDL_Init(SDL_INIT_VIDEO) < 0) {
+		printf("Could not init SDL\n");
+		exit(1);
+	}
+
+	atexit(SDL_Quit);
+
+	m->nx = 6;
+	m->ny = 4;
+
+	m->do_flash = 1;
+	m->do_focus = 0;
+	m->do_exposure = 1;
+	m->focus_min = 5;
+	m->do_white = 0;
+	m->do_big = 0;
+	m->do_full = 0;
+	m->zoom = 1;
+
+	m->window = SDL_CreateWindow("Camera", SDL_WINDOWPOS_UNDEFINED,
+				     SDL_WINDOWPOS_UNDEFINED, 800, 429,
+				     SDL_WINDOW_SHOWN | 
+				     m->do_full * SDL_WINDOW_FULLSCREEN_DESKTOP);
+	if (m->window == NULL) {
+		printf("Window could not be created! SDL_Error: %s\n", SDL_GetError());
+		exit(1);
+	}
+
+	sdl_init_window(m);
+}
+
+static void sdl_set_size(struct sdl *m, int _sx)
+{
+	m->sx = _sx;
+	m->factor = (float) m->dev->fmt.fmt.pix.width / m->sx;
+	m->sy = m->dev->fmt.fmt.pix.height / m->factor;
+
+	m->bx = (m->wx-m->sx)/2;
+	m->by = (m->wy-m->sy)/2;
+
+	m->liveview = SDL_CreateRGBSurface(0,m->sx,m->sy,32,0,0,0,0);
+	if (!m->liveview) {
+		printf("Couldn't create liveview\n");
+		exit(1);
+	}
+
+	sdl_paint_ui(m);
+	sdl_sync_settings(m);
+}
+
+static struct sdl sdl;
+
+static void cam_pgm_write(struct dev_info *dev, struct v4l2_format *fmt, unsigned char *img, char *out_name)
+{
+	char buf[1024];
+	double focus, gain, exposure;
+
+	exposure = v4l2_get_control(dev->fd, V4L2_CID_EXPOSURE_ABSOLUTE) / 65536.;
+	gain = v4l2_get_control(dev->fd, 0x00980913) / 65536.;
+	focus = v4l2_get_control(dev->fd, V4L2_CID_FOCUS_ABSOLUTE) / 65536.;
+
+	sprintf(buf, "# SDLcam raw image\n# Exposure %f, gain %f, focus %f\n", exposure, gain, focus);
+
+	pgm_write(fmt, img, out_name, buf);
+}
+
+#ifdef COOKED
+static void ppm_write(struct v4l2_format *fmt, unsigned char *img, char *out_name)
+{
+	FILE *fout;
+	fout = fopen(out_name, "w");
+	if (!fout) {
+		perror("Cannot open image");
+		exit(EXIT_FAILURE);
+	}
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_RGB24:
+		printf("ok\n");
+		break;
+	default:
+		printf("Bad pixel format\n");
+		exit(1);
+	}
+
+	fprintf(fout, "P6\n%d %d 255\n",
+		fmt->fmt.pix.width, fmt->fmt.pix.height);
+	fwrite(img, fmt->fmt.pix.width, 3*fmt->fmt.pix.height, fout);
+	fclose(fout);
+}
+
+/**
+   Write image to jpeg file.
+   \param img image to write
+*/
+static void jpeg_write(struct v4l2_format *fmt, unsigned char *img, char *jpegFilename)
+{
+	struct jpeg_compress_struct cinfo;
+	struct jpeg_error_mgr jerr;
+
+	JSAMPROW row_pointer[1];
+	FILE *outfile = fopen(jpegFilename, "wb");
+	if (!outfile) {
+		printf("Can't open jpeg for writing.\n");
+		exit(2);
+	}
+
+	/* create jpeg data */
+	cinfo.err = jpeg_std_error(&jerr);
+	jpeg_create_compress(&cinfo);
+	jpeg_stdio_dest(&cinfo, outfile);
+
+	/* set image parameters */
+	cinfo.image_width = fmt->fmt.pix.width;
+	cinfo.image_height = fmt->fmt.pix.height;
+	cinfo.input_components = 3;
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_RGB24:
+		cinfo.in_color_space = JCS_RGB;
+		break;
+	default:
+		printf("Need to specify colorspace!\n");
+		exit(1);
+	}
+
+	/* set jpeg compression parameters to default */
+	jpeg_set_defaults(&cinfo);
+	jpeg_set_quality(&cinfo, 90, TRUE);
+
+	jpeg_start_compress(&cinfo, TRUE);
+
+	/* feed data */
+	while (cinfo.next_scanline < cinfo.image_height) {
+		row_pointer[0] = &img[cinfo.next_scanline * cinfo.image_width *  cinfo.input_components];
+		jpeg_write_scanlines(&cinfo, row_pointer, 1);
+	}
+
+	jpeg_finish_compress(&cinfo);
+	jpeg_destroy_compress(&cinfo);
+	fclose(outfile);
+}
+
+static void convert_rgb(struct dev_info *dev, struct v4l2_format sfmt, void *buf, struct v4l2_format dfmt, void *buf2, int wb)
+{
+	struct v4lconvert_data *data = v4lconvert_create(dev->fd);
+	int res;
+
+	printf("Converting first.");
+	if (wb) {
+		struct v4l2_control ctrl;
+		ctrl.id = V4L2_CID_AUTO_WHITE_BALANCE;
+		ctrl.value = 1;
+		v4lconvert_vidioc_s_ctrl(data, &ctrl);
+	}
+	res = v4lconvert_convert(data, &sfmt, &dfmt, buf, SIZE, buf2, SIZE);
+	printf("Converting: %d\n", res);
+	v4lconvert_destroy(data);
+}
+#endif
+
+static void any_write(struct dev_info *dev)
+{
+	char name[1024];
+	unsigned char *buf;
+	time_t t = time(NULL);
+
+	buf = dev->buf;
+
+	if (1) {
+		sprintf(name, "/data/tmp/delme_%d.%s", (int) t, "pgm");
+
+		if (dev->fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_SGRBG10)
+			printf("Not in bayer10, can't write raw.\n");
+		else
+			cam_pgm_write(dev, &dev->fmt, buf, name);
+	}
+
+#ifdef COOKED
+	int ppm = 0;
+	/* FIXME: full resolution picture is too big to be put on stack */
+	static unsigned char buf2[SIZE];
+
+	sprintf(name, "/data/tmp/delme_%d.%s", (int) t, ppm ? "ppm" : "jpg");
+	if (dev->fmt.fmt.pix.pixelformat != V4L2_PIX_FMT_RGB24) {
+		struct v4l2_format dest_fmt;
+
+		dest_fmt = dev->fmt;
+		dest_fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
+		dest_fmt.fmt.pix.bytesperline = 3 * dest_fmt.fmt.pix.width;
+		dest_fmt.fmt.pix.sizeimage = 3 * dest_fmt.fmt.pix.width * dest_fmt.fmt.pix.height;
+
+		convert_rgb(dev, dev->fmt, buf, dest_fmt, buf2, 0);
+		buf = buf2;
+		//convert_rgb(dev, dest_fmt, buf2, dest_fmt, buf, 0);
+	}
+	if (ppm)
+		ppm_write(&dev->fmt, buf, name);
+	else
+		jpeg_write(&dev->fmt, buf, name);
+#endif
+}
+
+static void sdl_mouse(struct sdl *m, SDL_Event event)
+{
+	int ax = 0, ay = 0;
+	int nx = event.button.x / (m->wx/m->nx), ny = event.button.y / (m->wy/m->ny);
+	if (nx >= m->nx/2)
+		nx -= m->nx;
+	if (ny >= m->ny/2)
+		ny -= m->ny;
+
+	printf("Button %d %d\n", nx, ny);
+
+	/* Virtual slider */
+	if (nx == -2) {
+		double value = (double) event.button.y / m->wy;
+		switch (m->slider_mode) {
+		case M_BIAS: {
+			double ev = value - .5; /* -.5 .. +.5 */
+			ev *= 3000;
+			printf("Exposure bias: %f mEV %d\n", ev, (int) ev);
+			if (v4l2_s_ctrl(m->fd, V4L2_CID_AUTO_EXPOSURE_BIAS, ev) < 0) {
+				printf("Could not set exposure bias\n");
+			}
+		}
+			return;
+		case M_EXPOSURE:
+			m->do_exposure = 0;
+			cam_set_exposure(m, value);
+			sdl_sync_settings(m);
+			return;
+		case M_GAIN:
+			m->do_exposure = 0;
+			v4l2_set_control(m->fd, 0x00980913, value * 65535);
+			sdl_sync_settings(m);
+			return;
+		case M_FOCUS:
+			cam_set_focus(m, value);
+			return;
+		}
+	}
+
+	switch (ny) {
+	case 0:
+		switch (nx) {
+		case 0:
+			m->do_focus ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case -1:
+			m->do_exposure ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		}
+		break;
+	case 1:
+		switch (nx) {
+		case -1:
+			m->slider_mode = (m->slider_mode + 1) % M_NUM;
+			sdl_paint_ui(m);
+		}
+		break;
+	case -2:
+		switch (nx) {
+#if 0
+		case -2:
+			m->do_full ^= 1;
+			sdl_init_window(m);
+			sdl_paint_ui(m);
+			return;
+		case -1:
+			if (m->zoom == 1) {
+				m->zoom = 4;
+				return;
+			}
+			m->zoom = 1;
+			return;
+#endif
+		case -1:
+			v4l2_s_ctrl(m->fd, V4L2_CID_AUTO_FOCUS_STATUS, 1);
+			return;
+		break;
+		}
+	case -1:
+		switch (nx) {
+		case 0:
+			exit(0);
+		case 1:
+			m->do_flash ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case 2:
+			m->do_white ^= 1;
+			sdl_paint_ui(m);
+			sdl_sync_settings(m);
+			return;
+		case -3:
+			m->do_big ^= 1;
+			if (m->do_big)
+				sdl_set_size(m, m->do_full ? 630:512);
+			else
+				sdl_set_size(m, 256);
+			return;
+		case -1:
+			sdl_paint_ui(m);
+			any_write(m->dev);
+			return;
+		}
+		break;
+	}
+			
+	if (event.button.x > m->wx-m->bx)
+		ax = 1;
+	if (event.button.x < m->bx)
+		ax = -1;
+
+	if (event.button.y > m->wy-m->by)
+		ay = 1;
+	if (event.button.y < m->by)
+		ay = -1;
+	    
+	printf("mouse button at...%d, %d area %d, %d\n", event.button.x, event.button.y,
+	       ax, ay);
+}
+
+static void sdl_iteration(struct sdl *m)
+{
+	SDL_Event event;
+
+	while(SDL_PollEvent(&event)) {
+		switch(event.type) {
+		case SDL_QUIT:
+			exit(1);
+			break;
+		case SDL_KEYDOWN:
+			printf("key pressed... %c\n", event.key.keysym.sym);
+			/* SDLK_A, SDLK_LEFT, SDLK_RETURN, SDLK_BACKSPACE, SDLK_SPACE */
+			switch (event.key.keysym.sym) {
+			default: sdl_key(m, event.key.keysym.sym);
+			}
+			break;
+		case SDL_WINDOWEVENT:
+			if (event.window.event == SDL_WINDOWEVENT_EXPOSED)
+				sdl_paint_ui(m);
+			break;
+		case SDL_MOUSEBUTTONDOWN:
+			sdl_mouse(m, event);
+			break;
+		}
+	}
+}
+
+static void cam_open(struct dev_info *dev)
+{
+	struct v4l2_format *fmt = &dev->fmt;
+
+	dev->fd = v4l2_open("/dev/video_ccdc", O_RDWR);
+
+	fmt->type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
+	fmt->fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG10;
+//	fmt->fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
+	fmt->fmt.pix.field = V4L2_FIELD_NONE;
+	fmt->fmt.pix.width = SX;
+	fmt->fmt.pix.height = SY;
+
+	v4l2_s_ctrl(dev->fd, V4L2_CID_AUTO_FOCUS_STATUS, 0);
+	v4l2_set_focus(dev->fd, 50);
+
+	printf("ioctl = %d\n", v4l2_ioctl(dev->fd, VIDIOC_S_FMT, fmt));
+
+	printf("capture is %lx %s %d x %d\n", (unsigned long) fmt->fmt.pix.pixelformat, fmt_name(fmt), fmt->fmt.pix.width, fmt->fmt.pix.height);
+}
+
+/* ------------------------------------------------------------------ */
+
+
+static struct dev_info dev;
+
+int main(void)
+{
+	int i;
+	struct v4l2_format *fmt = &dev.fmt;
+
+	dtime();
+	cam_open(&dev);
+
+	/* factor == 3 still fits window, but very slow over ssh. (Fast enough on localhost) */
+	/* factor == 5 nice over ssh */
+	/* == 1.7, full-screen video for 800x600 */
+	/* == 2.5, full-screen video for 1200x, 516 */
+	sdl_init(&sdl, &dev);
+	sdl_set_size(&sdl, 256);
+  
+	/* In 800x600 "raw" mode, this should take cca 17.8 seconds (without
+	   sdl output. CPU usage should be cca 5% without conversion). That's 28 fps.
+	   (benchmark with i<500)
+
+	   switched to separate application -- not in library:
+	   1296x986, i<20, with RGB24 conversion: 32 sec
+	   1296x986, i<20, raw _SGRBG10: 6.3 sec
+
+	   i<500, 2m59, 86%, RGB24
+	   i<500, 1m43, 36%, b10
+	*/
+	double loop = dtime(), max = 0, avg = .200;
+	if (dev.debug & D_TIMING)
+		printf("startup took %f\n", loop);
+	
+	for (i=0; i<500000; i++) {
+		int num = v4l2_read(dev.fd, dev.buf, SIZE);
+		if (num < 0)
+			return 1;
+		/* Over USB connection, rendering every single frame slows
+		   execution down from 23 seconds to 36 seconds. */
+#if 1
+		{
+			/* Render takes 80msec over ssh, 47msec on localhost */
+			double d = dtime();
+			sdl_render(&sdl, dev.buf, fmt);
+			if (dev.debug & D_TIMING)
+				printf("Render took %f\n", dtime() - d);
+		}
+		{
+			/* Takes cca 10msec over ssh */
+			double d = dtime();
+			for (int i = 0; i<1; i++)
+				sdl_status(&sdl, avg);
+			if (dev.debug & D_TIMING)
+				printf("Status took %f\n", dtime() - d);
+		}
+#endif
+		sdl_iteration(&sdl);
+		double now = dtime();
+		if (now - loop > max)
+			max = now - loop;
+		double c = 0.03;
+		avg = (now - loop) * c + avg * (1-c);
+		if (dev.debug & D_TIMING)
+			printf("Iteration %f, maximum %f, average %f\n", now-loop, max, avg);
+		loop = now;
+	}
+	return 0;
+}
diff --git a/contrib/test/unused-v4l2sdl.c b/contrib/test/unused-v4l2sdl.c
new file mode 100644
index 0000000..fe79b9e
--- /dev/null
+++ b/contrib/test/unused-v4l2sdl.c
@@ -0,0 +1,124 @@
+/*
+
+gcc sdlcam.c -I. -I../../include -I../../lib/include -I../../lib/libv4l2 -I../../lib/libv4lconvert/processing -I../../lib/libv4lconvert -std=c99 -lSDL2 -lSDL2_image -ljpeg -lefence ../../lib/libv4l2/.libs/libv4l2.a  ../../lib/libv4lconvert/.libs/libv4lconvert.a && time ./a.out
+
+  libv4l is problematic:
+  1) It converts image to RGB24, but we'd prefer to stay in GRBG10
+  2) It only uses 8-bits per color internally. Cameras such as Nikon D500 
+     store 14-bits per channel. Even N900 stores 10.
+  3) We'd really like to know units for exposure/gain/focus.
+  4) It does not work with subdevices -- at all.
+  5) It steps up time before stepping up gain -- unsuitable for camera.
+  6) Too much resolution in time at the high end, and not enough at the low end, thus oscillations
+
+*/
+
+void imageStatistics(FCam::Image img)
+{
+	int sx, sy;
+
+	sx = img.size().width;                                                          
+	sy = img.size().height;                                                         
+	printf("Image is %d x %d, ", sx, sy);                                           
+
+	printf("(%d) ", img.brightness(sx/2, sy/2));
+
+	int dark = 0;                                                                   
+	int bright = 0;                                                                 
+	int total = 0;                                                                  
+#define RATIO 10
+	for (int y = sy/(2*RATIO); y < sy; y += sy/RATIO)                               
+		for (int x = sx/(2*RATIO); x < sx; x += sx/RATIO) {                    
+			int br = img.brightness(x, y);                               
+
+			if (!d_raw) {
+				/* It seems real range is 60 to 218 */
+				if (br > 200)
+					bright++;                                               
+				if (br < 80)
+					dark++;
+			} else {
+				/* there's a lot of noise, it seems black is commonly 65..71,
+				   bright is cca 1023 */
+				if (br > 1000)
+					bright++;
+				if (br < 75)
+					dark++;
+			}
+			total++;                                                      
+		}
+
+	printf("%d dark %d bri,", dark, bright);                     
+
+	long sharp = 0;
+	for (int y = sy/3; y < 2*(sy/3); y+=sy/12) {
+		int b = -1;
+		for (int x = sx/3; x < 2*(sx/3); x++) {
+			int b2;                                                                
+			b2 = img.brightness(x, y/2);                                          
+			if (b != -1)
+				sharp += (b-b2) * (b-b2);                                              
+			b = b2;                                                                
+		}
+	}
+	printf("sh %d\n", sharp);
+}
+
+static void v4l2_statistics(int index, struct v4l2_buffer *buf)
+{
+	unsigned char *b;
+	struct v4l2_format *fmt;
+
+	fmt = &devices[index].src_fmt;
+	b = devices[index].frame_pointers[buf->index];
+
+	if (!(devices[index].frame%3))
+		v4l2_auto_exposure(index, buf);
+	
+	int sh = v4l2_sharpness(b, fmt);
+	v4l2_auto_focus_single(&devices[index], sh);
+	
+	devices[index].frame++;
+	if (!(devices[index].frame%4))
+		sdl_render(&sdl, b, fmt);
+}
+
+static int v4l2_set_autogain(int fd)
+{
+	if (v4l2_s_ctrl(fd, V4L2_CID_AUTOGAIN, 1) < 0) {
+		printf("Could not set autogain\n");
+	}
+	return 0;
+}
+
+static int v4l2_set_exposure(int fd, int exposure)
+{
+	if (v4l2_s_ctrl(fd, V4L2_CID_EXPOSURE_ABSOLUTE, exposure) < 0) {
+		printf("Could not set exposure\n");
+	}
+	return 0;
+}
+
+static int v4l2_set_gain(int fd, int gain)
+{
+	if (v4l2_s_ctrl(fd, 0x00980913, gain) < 0) {
+		printf("Could not set gain\n");
+	}
+	return 0;
+}
+
+static double gain_to_EV(double v)
+{
+	return -(log(v)/log(2));
+}
+
+static double EV_to_lux(double v)
+{
+	return 2.5 * pow(2, v);
+}
+
+/* Takes inverse 1/time! */
+static double time_to_EV(double v) {
+	return 5+log(v)/log(2);
+}
+
diff --git a/contrib/test/v4l2grab.c b/contrib/test/v4l2grab.c
index 778c51c..8b25b45 100644
--- a/contrib/test/v4l2grab.c
+++ b/contrib/test/v4l2grab.c
@@ -326,7 +326,7 @@ static int capture(char *dev_name, int x_res, int y_res, int n_frames,
  * capture routine
  */
 
-const char *argp_program_version = "V4L2 grabber version " V4L_UTILS_VERSION;
+const char *argp_program_version = "V4L2 grabber version <local>";
 const char *argp_program_bug_address = "Mauro Carvalho Chehab <m.chehab@samsung.com>";
 
 static const char doc[] = "\nCapture images using libv4l, storing them as ppm files\n";
diff --git a/lib/libv4l2/libv4l2-priv.h b/lib/libv4l2/libv4l2-priv.h
index 343db5e..a6bc48e 100644
--- a/lib/libv4l2/libv4l2-priv.h
+++ b/lib/libv4l2/libv4l2-priv.h
@@ -26,6 +26,7 @@
 #include "../libv4lconvert/libv4lsyscall-priv.h"
 
 #define V4L2_MAX_DEVICES 16
+#define V4L2_MAX_SUBDEVS 8
 /* Warning when making this larger the frame_queued and frame_mapped members of
    the v4l2_dev_info struct can no longer be a bitfield, so the code needs to
    be adjusted! */
@@ -104,6 +105,7 @@ struct v4l2_dev_info {
 	void *plugin_library;
 	void *dev_ops_priv;
 	const struct libv4l_dev_ops *dev_ops;
+        int subdev_fds[V4L2_MAX_SUBDEVS];
 };
 
 /* From v4l2-plugin.c */
diff --git a/lib/libv4l2/libv4l2.c b/lib/libv4l2/libv4l2.c
index 2db25d1..f9e2711 100644
--- a/lib/libv4l2/libv4l2.c
+++ b/lib/libv4l2/libv4l2.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008 Hans de Goede <hdegoede@redhat.com>
 
@@ -794,13 +795,21 @@ no_capture:
 		v4lconvert_set_fps(devices[index].convert, V4L2_DEFAULT_FPS);
 	v4l2_update_fps(index, &parm);
 
+	devices[index].subdev_fds[0] = SYS_OPEN("/dev/video_sensor", O_RDWR, 0);
+	devices[index].subdev_fds[1] = SYS_OPEN("/dev/video_focus", O_RDWR, 0);
+	devices[index].subdev_fds[2] = SYS_OPEN("/dev/video_flash", O_RDWR, 0);
+	devices[index].subdev_fds[3] = -1;
+
+	printf("Sensor: %d, focus: %d\n", devices[index].subdev_fds[0], 
+	       devices[index].subdev_fds[1]);
+
 	V4L2_LOG("open: %d\n", fd);
 
 	return fd;
 }
 
 /* Is this an fd for which we are emulating v4l1 ? */
-static int v4l2_get_index(int fd)
+int v4l2_get_index(int fd)
 {
 	int index;
 
@@ -1056,6 +1065,21 @@ static int v4l2_s_fmt(int index, struct v4l2_format *dest_fmt)
 	return 0;
 }
 
+static int v4l2_propagate_ioctl(int index, unsigned long request, void *arg)
+{
+	int i = 0;
+	int result;
+	while (1) {
+		if (devices[index].subdev_fds[i] == -1)
+			return -1;
+		result = SYS_IOCTL(devices[index].subdev_fds[i], request, arg);
+		if (result == 0)
+			return 0;
+		i++;
+	}
+	return -1;
+}
+
 int v4l2_ioctl(int fd, unsigned long int request, ...)
 {
 	void *arg;
@@ -1185,14 +1209,20 @@ no_capture_request:
 	switch (request) {
 	case VIDIOC_QUERYCTRL:
 		result = v4lconvert_vidioc_queryctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_CTRL:
 		result = v4lconvert_vidioc_g_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_S_CTRL:
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, arg);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, request, arg);
 		break;
 
 	case VIDIOC_G_EXT_CTRLS:
@@ -1739,7 +1769,8 @@ int v4l2_set_control(int fd, int cid, int value)
 
 	result = v4lconvert_vidioc_queryctrl(devices[index].convert, &qctrl);
 	if (result)
-		return result;
+		if (v4l2_propagate_ioctl(index, VIDIOC_QUERYCTRL, &qctrl))
+			return -1;
 
 	if (!(qctrl.flags & V4L2_CTRL_FLAG_DISABLED) &&
 			!(qctrl.flags & V4L2_CTRL_FLAG_GRABBED)) {
@@ -1750,6 +1781,8 @@ int v4l2_set_control(int fd, int cid, int value)
 				qctrl.minimum;
 
 		result = v4lconvert_vidioc_s_ctrl(devices[index].convert, &ctrl);
+		if (result == -1)
+			result = v4l2_propagate_ioctl(index, VIDIOC_S_CTRL, &ctrl);
 	}
 
 	return result;
@@ -1768,7 +1801,8 @@ int v4l2_get_control(int fd, int cid)
 	}
 
 	if (v4lconvert_vidioc_queryctrl(devices[index].convert, &qctrl))
-		return -1;
+		if (v4l2_propagate_ioctl(index, VIDIOC_QUERYCTRL, &qctrl))
+			return -1;
 
 	if (qctrl.flags & V4L2_CTRL_FLAG_DISABLED) {
 		errno = EINVAL;
diff --git a/lib/libv4lconvert/Makefile.am b/lib/libv4lconvert/Makefile.am
index f266f3e..9557a9c 100644
--- a/lib/libv4lconvert/Makefile.am
+++ b/lib/libv4lconvert/Makefile.am
@@ -17,6 +17,7 @@ libv4lconvert_la_SOURCES = \
   stv0680.c cpia1.c se401.c jpgl.c jpeg.c jl2005bcd.c \
   control/libv4lcontrol.c control/libv4lcontrol.h control/libv4lcontrol-priv.h \
   processing/libv4lprocessing.c processing/whitebalance.c processing/autogain.c \
+  processing/focus.c \
   processing/gamma.c processing/libv4lprocessing.h processing/libv4lprocessing-priv.h \
   helper-funcs.h libv4lconvert-priv.h libv4lsyscall-priv.h \
   tinyjpeg.h tinyjpeg-internal.h
diff --git a/lib/libv4lconvert/control/libv4lcontrol.c b/lib/libv4lconvert/control/libv4lcontrol.c
index 1e784ed..e8c67e2 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.c
+++ b/lib/libv4lconvert/control/libv4lcontrol.c
@@ -638,7 +638,7 @@ struct v4lcontrol_data *v4lcontrol_create(int fd, void *dev_ops_priv,
 
 	/* If the device always needs conversion, we can add fake controls at no cost
 	   (no cost when not activated by the user that is) */
-	if (always_needs_conversion || v4lcontrol_needs_conversion(data)) {
+	if (1 || always_needs_conversion || v4lcontrol_needs_conversion(data)) {
 		for (i = 0; i < V4LCONTROL_AUTO_ENABLE_COUNT; i++) {
 			ctrl.id = fake_controls[i].id;
 			rc = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
@@ -678,6 +678,16 @@ struct v4lcontrol_data *v4lcontrol_create(int fd, void *dev_ops_priv,
 		break;
 	}
 
+       data->controls |= 1 << V4LCONTROL_AUTOGAIN |
+	 	         1 << V4LCONTROL_AUTOGAIN_TARGET |
+	                 1 << V4LCONTROL_EXPOSURE_BIAS |
+	 	         1 << V4LCONTROL_AUTO_FOCUS |
+	 	         1 << V4LCONTROL_FOCUS_START |
+	 		 1 << V4LCONTROL_FOCUS_STOP |
+	 		 1 << V4LCONTROL_FOCUS_STATUS |
+	 		 1 << V4LCONTROL_FOCUS_RANGE;
+
+
 	/* Allow overriding through environment */
 	s = getenv("LIBV4LCONTROL_CONTROLS");
 	if (s)
@@ -841,6 +851,60 @@ static const struct v4l2_queryctrl fake_controls[V4LCONTROL_COUNT] = {
 		.step = 1,
 		.default_value = 100,
 		.flags = V4L2_CTRL_FLAG_SLIDER
+	},  {
+	        .id = V4L2_CID_FOCUS_AUTO,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "Auto Focus enabled",
+		.minimum = 0,
+		.maximum = 1,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
+	},  {
+	        .id = V4L2_CID_AUTO_FOCUS_START,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "Focus start",
+		.minimum = 0,
+		.maximum = 1,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
+	},  {
+	        .id = V4L2_CID_AUTO_FOCUS_STOP,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "",
+		.minimum = 0,
+		.maximum = 1,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
+	},  {
+	        .id = V4L2_CID_AUTO_FOCUS_STATUS,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "",
+		.minimum = 0,
+		.maximum = 1,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
+	}, {
+	        .id = V4L2_CID_AUTO_FOCUS_RANGE,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "",
+		.minimum = 0,
+		.maximum = 1,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
+	}, {
+	        .id = V4L2_CID_AUTO_EXPOSURE_BIAS,
+		.type = V4L2_CTRL_TYPE_INTEGER,
+		.name =  "",
+		.minimum = -10000,
+		.maximum = 10000,
+		.step = 1,
+		.default_value = 0,
+		.flags = 0
 	},
 };
 
@@ -905,6 +969,7 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 {
 	int i;
 	struct v4l2_control *ctrl = arg;
+	int res;
 
 	for (i = 0; i < V4LCONTROL_COUNT; i++)
 		if ((data->controls & (1 << i)) &&
@@ -913,8 +978,10 @@ int v4lcontrol_vidioc_g_ctrl(struct v4lcontrol_data *data, void *arg)
 			return 0;
 		}
 
-	return data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
+	res = data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
 			VIDIOC_G_CTRL, arg);
+
+	return res;
 }
 
 static void v4lcontrol_alloc_valid_controls(struct v4lcontrol_data *data,
diff --git a/lib/libv4lconvert/control/libv4lcontrol.h b/lib/libv4lconvert/control/libv4lcontrol.h
index fa9cf42..7a970cb 100644
--- a/lib/libv4lconvert/control/libv4lcontrol.h
+++ b/lib/libv4lconvert/control/libv4lcontrol.h
@@ -45,6 +45,12 @@ enum {
 	V4LCONTROL_AUTO_ENABLE_COUNT,
 	V4LCONTROL_AUTOGAIN,
 	V4LCONTROL_AUTOGAIN_TARGET,
+	V4LCONTROL_AUTO_FOCUS,
+	V4LCONTROL_FOCUS_START,
+	V4LCONTROL_FOCUS_STOP,
+	V4LCONTROL_FOCUS_STATUS,
+	V4LCONTROL_FOCUS_RANGE,
+	V4LCONTROL_EXPOSURE_BIAS,
 	V4LCONTROL_COUNT
 };
 
diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
index 1a5ccec..e1db987 100644
--- a/lib/libv4lconvert/libv4lconvert.c
+++ b/lib/libv4lconvert/libv4lconvert.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008-2011 Hans de Goede <hdegoede@redhat.com>
 
@@ -71,6 +72,8 @@ const struct libv4l_dev_ops *v4lconvert_get_default_dev_ops()
 	return &default_dev_ops;
 }
 
+#define V4LCONVERT_ERR printf
+
 static void v4lconvert_get_framesizes(struct v4lconvert_data *data,
 		unsigned int pixelformat, int index);
 
@@ -89,7 +92,8 @@ static void v4lconvert_get_framesizes(struct v4lconvert_data *data,
 	{ V4L2_PIX_FMT_RGB24,		24,	 1,	 5,	0 }, \
 	{ V4L2_PIX_FMT_BGR24,		24,	 1,	 5,	0 }, \
 	{ V4L2_PIX_FMT_YUV420,		12,	 6,	 1,	0 }, \
-	{ V4L2_PIX_FMT_YVU420,		12,	 6,	 1,	0 }
+	{ V4L2_PIX_FMT_YVU420,		12,	 6,	 1,	0 }, \
+	{ V4L2_PIX_FMT_SGRBG10,		16,	 8,	 8,	0 }
 
 static const struct v4lconvert_pixfmt supported_src_pixfmts[] = {
 	SUPPORTED_DST_PIXFMTS,
@@ -131,7 +135,7 @@ static const struct v4lconvert_pixfmt supported_src_pixfmts[] = {
 	{ V4L2_PIX_FMT_SGBRG8,		 8,	 8,	 8,	0 },
 	{ V4L2_PIX_FMT_SGRBG8,		 8,	 8,	 8,	0 },
 	{ V4L2_PIX_FMT_SRGGB8,		 8,	 8,	 8,	0 },
-	{ V4L2_PIX_FMT_STV0680,		 8,	 8,	 8,	1 },
+	{ V4L2_PIX_FMT_STV0680,		 8,	 8,	 8,	0 },
 	{ V4L2_PIX_FMT_SGRBG10,		16,	 8,	 8,	1 },
 	/* compressed bayer */
 	{ V4L2_PIX_FMT_SPCA561,		 0,	 9,	 9,	1 },
@@ -206,14 +210,16 @@ struct v4lconvert_data *v4lconvert_create_with_dev_ops(int fd, void *dev_ops_pri
 	data->fps = 30;
 
 	/* Check supported formats */
-	for (i = 0; ; i++) {
+	for (i = 0; i < 1; i++) {
 		struct v4l2_fmtdesc fmt = { .type = V4L2_BUF_TYPE_VIDEO_CAPTURE };
 
 		fmt.index = i;
 
 		if (data->dev_ops->ioctl(data->dev_ops_priv, data->fd,
-				VIDIOC_ENUM_FMT, &fmt))
-			break;
+					 VIDIOC_ENUM_FMT, &fmt)) {
+			printf("Enum fmt failed. Assuming N900: SGRBG10\n");
+			fmt.pixelformat = V4L2_PIX_FMT_SGRBG10;
+		}
 
 		for (j = 0; j < ARRAY_SIZE(supported_src_pixfmts); j++)
 			if (fmt.pixelformat == supported_src_pixfmts[j].fmt)
@@ -371,6 +377,7 @@ static int v4lconvert_get_rank(struct v4lconvert_data *data,
 	int needed, rank = 0;
 
 	switch (dest_pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:		
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		rank = supported_src_pixfmts[src_index].rgb_rank;
@@ -525,6 +532,10 @@ static int v4lconvert_do_try_format(struct v4lconvert_data *data,
 void v4lconvert_fixup_fmt(struct v4l2_format *fmt)
 {
 	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10:
+		fmt->fmt.pix.bytesperline = fmt->fmt.pix.width * 2;
+		fmt->fmt.pix.sizeimage = fmt->fmt.pix.width * fmt->fmt.pix.height * 2;
+		break;
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		fmt->fmt.pix.bytesperline = fmt->fmt.pix.width * 3;
@@ -687,6 +698,7 @@ static int v4lconvert_processing_needs_double_conversion(
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SRGGB8:
 	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10:
 		return 0;
 	}
 	switch (dest_pix_fmt) {
@@ -740,6 +752,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 	unsigned int height = fmt->fmt.pix.height;
 	unsigned int bytesperline = fmt->fmt.pix.bytesperline;
 
+	//printf("Convert_pixfmt %lx -> %lx\n", src_pix_fmt, dest_pix_fmt);
+	
 	switch (src_pix_fmt) {
 	/* JPG and variants */
 	case V4L2_PIX_FMT_MJPEG:
@@ -866,6 +880,9 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			v4lconvert_yuv420_to_bgr24(data->convert_pixfmt_buf, dest, width,
 					height, yvu);
 			break;
+		case V4L2_PIX_FMT_SGRBG10:
+			printf("Convert yuv to sgrbg10\n");
+			exit (1);
 		}
 		break;
 	}
@@ -885,6 +902,9 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 		case V4L2_PIX_FMT_YVU420:
 			v4lconvert_hm12_to_yuv420(src, dest, width, height, 1);
 			break;
+		case V4L2_PIX_FMT_SGRBG10:
+			printf("Convert something to sgrbg10\n");
+			exit (1);
 		}
 		break;
 
@@ -903,6 +923,12 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 		unsigned char *tmpbuf;
 		struct v4l2_format tmpfmt = *fmt;
 
+		if (src_pix_fmt == dest_pix_fmt) {
+			int to_copy = MIN(dest_size, src_size);
+			memcpy(dest, src, to_copy);
+			return to_copy;
+		}
+
 		tmpbuf = v4lconvert_alloc_buffer(width * height,
 				&data->convert_pixfmt_buf, &data->convert_pixfmt_buf_size);
 		if (!tmpbuf)
@@ -986,6 +1012,7 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			V4LCONVERT_ERR("short raw bayer data frame\n");
 			errno = EPIPE;
 			result = -1;
+			/* FIXME: but then we proceed anyway?! */
 		}
 		switch (dest_pix_fmt) {
 		case V4L2_PIX_FMT_RGB24:
@@ -1000,6 +1027,9 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 		case V4L2_PIX_FMT_YVU420:
 			v4lconvert_bayer_to_yuv420(src, dest, width, height, bytesperline, src_pix_fmt, 1);
 			break;
+		case V4L2_PIX_FMT_SGRBG10:
+			printf("Convert bayer to sgrbg10\n");
+			exit(1);
 		}
 		break;
 
@@ -1485,6 +1515,12 @@ int v4lconvert_convert(struct v4lconvert_data *data,
 		temp_needed =
 			my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 3 / 2;
 		break;
+	case V4L2_PIX_FMT_SGRBG10:
+		dest_needed =
+			my_dest_fmt.fmt.pix.width * my_dest_fmt.fmt.pix.height * 2;
+		temp_needed =
+			my_src_fmt.fmt.pix.width * my_src_fmt.fmt.pix.height * 2;
+		break;
 	default:
 		V4LCONVERT_ERR("Unknown dest format in conversion\n");
 		errno = EINVAL;
diff --git a/lib/libv4lconvert/processing/autogain.c b/lib/libv4lconvert/processing/autogain.c
index c6866d6..0888697 100644
--- a/lib/libv4lconvert/processing/autogain.c
+++ b/lib/libv4lconvert/processing/autogain.c
@@ -1,3 +1,4 @@
+/* -*- c-file-style: "linux" -*- */
 /*
 #             (C) 2008-2009 Hans de Goede <hdegoede@redhat.com>
 
@@ -21,6 +22,7 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <unistd.h>
+#include <math.h>
 
 #include "libv4lprocessing.h"
 #include "libv4lprocessing-priv.h"
@@ -37,6 +39,20 @@ static int autogain_active(struct v4lprocessing_data *data)
 		data->last_gain_correction = 0;
 	}
 
+	/* Related controls:
+
+	   V4L2_CID_EXPOSURE_ABSOLUTE (integer)
+	   V4L2_CID_EXPOSURE_AUTO_PRIORITY
+	   V4L2_CID_EXPOSURE_BIAS ... mEV
+	   V4L2_CID_EXPOSURE_METERING
+
+	   V4L2_CID_ISO_SENSITIVITY
+	   V4L2_CID_ISO_SENSITIVITY_AUTO
+
+	   V4L2_CID_SCENE_MODE
+	   ...sports.
+	*/
+
 	return autogain;
 }
 
@@ -46,6 +62,8 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
 {
 	int ctrl_range = (ctrl->maximum - ctrl->minimum) / ctrl->step;
 
+	printf("Audogain: adjust\n");
+
 	/* If we are of 3 * deadzone or more, and we have a fine grained
 	   control, take larger steps, otherwise we take ages to get to the
 	   right setting point. We use 256 as tripping point for determining
@@ -68,6 +86,196 @@ static void autogain_adjust(struct v4l2_queryctrl *ctrl, int *value,
 	}
 }
 
+static int get_luminosity_bayer10(uint16_t *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 8 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline / 2 - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	avg_lum /= 4;
+	return avg_lum;
+}
+
+static int get_luminosity_bayer8(unsigned char *buf, const struct v4l2_format *fmt)
+{
+	long long avg_lum = 0;
+	int x, y;
+	
+	buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
+		fmt->fmt.pix.width / 4;
+
+	for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+		for (x = 0; x < fmt->fmt.pix.width / 2; x++)
+			avg_lum += *buf++;
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
+	}
+	avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+	return avg_lum;
+}
+
+#define BUCKETS 20
+
+static void v4l2_histogram_bayer10(unsigned short *buf, int cdf[], const struct v4l2_format *fmt)
+{
+    for (int y = 0; y < fmt->fmt.pix.height; y+=19)
+      for (int x = 0; x < fmt->fmt.pix.width; x+=19) {
+	int b;
+	b = buf[fmt->fmt.pix.width*y + x];
+	b = (b * BUCKETS)/(1024);
+	cdf[b]++;
+      }
+}
+
+static int v4l2_s_ctrl(int fd, long id, long value)
+{
+        int res;
+	struct v4l2_control ctrl;
+        ctrl.id = id;
+	ctrl.value = value;
+        res = v4l2_ioctl(fd, VIDIOC_S_CTRL, &ctrl);
+        if (res < 0)
+                printf("Set control %lx %ld failed\n", id, value);
+        return res;
+}
+
+static double get_bias(struct v4lprocessing_data *data)
+{
+	double bias;
+
+	bias = v4lcontrol_get_ctrl(data->control, V4LCONTROL_EXPOSURE_BIAS);
+	printf("Control bias %f\n", bias);
+#if 1
+	bias = bias / 1000;
+	bias = exp2(bias); /* EV -> internal multipliers */
+#endif
+	return bias;
+}
+
+static int v4l2_set_exposure(struct v4lprocessing_data *data, double exposure)
+{
+	double exp, gain; /* microseconds */
+	int exp_, gain_;
+	int fd = data->fd;
+
+	gain = 1;
+	exp = exposure / gain;
+	if (exp > 10000) {
+		exp = 10000;
+		gain = exposure / exp;
+	}
+	if (gain > 16) {
+		gain = 16;
+		exp = exposure / gain;
+	}
+
+	exp_ = exp;
+	gain_ = 10*log(gain)/log(2); 
+	printf("Exposure %f %d, gain %f %d\n", exp, exp_, gain, gain_);
+
+	/* gain | ISO | gain_ */
+	/* 1.   | 100 | 0 */
+	/* 2.   | 200 | 10 */
+
+        /* 16.   | 1600 | 40 */
+			
+	if (v4l2_s_ctrl(fd, 0x00980913 /* FIXME? V4L2_CID_GAIN */, gain_) < 0) {
+		printf("Could not set gain\n");
+	}
+	if (v4l2_s_ctrl(fd, V4L2_CID_EXPOSURE_ABSOLUTE, exp_) < 0) {
+		printf("Could not set exposure\n");
+	}
+	return 0;
+}
+
+struct exposure_data {
+	double exposure;
+};
+
+static int autogain_calculate_lookup_tables_exp(
+		struct v4lprocessing_data *data,
+		unsigned char *buf, const struct v4l2_format *fmt)
+{
+	int cdf[BUCKETS] = { 0, };
+	static struct exposure_data e_data;
+	static struct exposure_data *exp = &e_data;
+
+	v4l2_histogram_bayer10((void *) buf, cdf, fmt);
+
+#if 0
+	printf("hist: ");
+	for (i = 0; i<BUCKETS; i++)
+		printf("%d ", cdf[i]);
+	printf("\n");
+#endif
+	for (int i=1; i<BUCKETS; i++)
+		cdf[i] += cdf[i-1];
+
+	int b = BUCKETS;
+	int brightPixels = cdf[b-1] - cdf[b-8];
+	int targetBrightPixels = cdf[b-1]/50;
+	int maxSaturatedPixels = cdf[b-1]/200;
+	int saturatedPixels = cdf[b-1] - cdf[b-2];
+	// how much should I change brightness by
+	float adjustment = 1.0f;
+#if 0
+	printf( "AutoExposure: totalPixels: %d,"
+		"brightPixels: %d, targetBrightPixels: %d,"
+		"saturatedPixels: %d, maxSaturatedPixels: %d\n",
+		cdf[b-1], brightPixels, targetBrightPixels,
+		saturatedPixels, maxSaturatedPixels);
+#endif
+	  
+	if (saturatedPixels > maxSaturatedPixels) {
+		// first don't let things saturate too much
+		adjustment = 1.0f - ((float)(saturatedPixels - maxSaturatedPixels))/cdf[b-1];
+	} else if (brightPixels < (targetBrightPixels - (saturatedPixels * 4))) {
+		// increase brightness to try and hit the desired number of well exposed pixels
+		int l = b-6;
+		while (brightPixels < targetBrightPixels && l > 0) {
+			brightPixels += cdf[l];
+			brightPixels -= cdf[l-1];
+			l--;
+		}
+
+		// that level is supposed to be at b-11;
+		adjustment = ((float) (b-6+1))/(l+1);
+	} else {
+		// we're not oversaturated, and we have enough bright pixels. Do nothing.
+	}
+
+	{
+		float limit = 4;
+		if (adjustment > limit) { adjustment = limit; }
+		if (adjustment < 1/limit) { adjustment = 1/limit; }
+	}
+	adjustment *= get_bias(data);
+	  
+	exp->exposure *= adjustment;
+	if (exp->exposure < 1)
+		exp->exposure = 1;
+	{
+		float limit = 64000000;
+		if (exp->exposure > limit)
+			exp->exposure = limit;
+	}
+	
+	if (adjustment != 1.)
+		printf( "AutoExposure: adjustment: %f exposure %f\n", adjustment, exp->exposure);
+
+	v4l2_set_exposure(data, exp->exposure);
+	return 0;
+}
+
+/* This autogain version is suitable for webcams with linear controls, not for cameras */
+
 /* auto gain and exposure algorithm based on the knee algorithm described here:
 http://ytse.tricolour.net/docs/LowLightOptimization.html */
 static int autogain_calculate_lookup_tables(
@@ -80,11 +288,17 @@ static int autogain_calculate_lookup_tables(
 	struct v4l2_queryctrl gainctrl, expoctrl;
 	const int deadzone = 6;
 
+	printf("autogain: calculate lookups\n");
+
 	ctrl.id = V4L2_CID_EXPOSURE;
 	expoctrl.id = V4L2_CID_EXPOSURE;
-	if (SYS_IOCTL(data->fd, VIDIOC_QUERYCTRL, &expoctrl) ||
-			SYS_IOCTL(data->fd, VIDIOC_G_CTRL, &ctrl))
-		return 0;
+	if (v4l2_ioctl(data->fd, VIDIOC_QUERYCTRL, &expoctrl) ||
+	    v4l2_ioctl(data->fd, VIDIOC_G_CTRL, &ctrl)) {
+		/* No exposure control; but perhaps this is digital camera,
+		   we have V4L2_CID_EXPOSURE_ABSOLUTE, and should use different
+		   control algorithm. */
+		return autogain_calculate_lookup_tables_exp(data, buf, fmt);
+	}
 
 	exposure = orig_exposure = ctrl.value;
 	/* Determine a value below which we try to not lower the exposure,
@@ -100,25 +314,26 @@ static int autogain_calculate_lookup_tables(
 
 	ctrl.id = V4L2_CID_GAIN;
 	gainctrl.id = V4L2_CID_GAIN;
-	if (SYS_IOCTL(data->fd, VIDIOC_QUERYCTRL, &gainctrl) ||
-			SYS_IOCTL(data->fd, VIDIOC_G_CTRL, &ctrl))
+	if (v4l2_ioctl(data->fd, VIDIOC_QUERYCTRL, &gainctrl) ||
+	    v4l2_ioctl(data->fd, VIDIOC_G_CTRL, &ctrl))
 		return 0;
 	gain = orig_gain = ctrl.value;
 
+	printf("Looking at pixels...\n");
+
 	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
+		avg_lum = get_luminosity_bayer10((void *) buf, fmt);
+		break;
+
 	case V4L2_PIX_FMT_SGBRG8:
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
-		buf += fmt->fmt.pix.height * fmt->fmt.pix.bytesperline / 4 +
-			fmt->fmt.pix.width / 4;
-
-		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
-			for (x = 0; x < fmt->fmt.pix.width / 2; x++)
-				avg_lum += *buf++;
-			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width / 2;
-		}
-		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width / 4;
+		avg_lum = get_luminosity_bayer8(buf, fmt);
 		break;
 
 	case V4L2_PIX_FMT_RGB24:
@@ -137,7 +352,7 @@ static int autogain_calculate_lookup_tables(
 		avg_lum /= fmt->fmt.pix.height * fmt->fmt.pix.width * 3 / 4;
 		break;
 	}
-
+	
 	/* If we are off a multiple of deadzone, do multiple steps to reach the
 	   desired lumination fast (with the risc of a slight overshoot) */
 	target = v4lcontrol_get_ctrl(data->control, V4LCONTROL_AUTOGAIN_TARGET);
@@ -199,20 +414,23 @@ static int autogain_calculate_lookup_tables(
 		data->lookup_table_update_counter = V4L2PROCESSING_UPDATE_RATE - 2;
 	}
 
+	printf("Setting gain %d / exposure %d\n", gain, exposure);
+
 	if (gain != orig_gain) {
 		ctrl.id = V4L2_CID_GAIN;
 		ctrl.value = gain;
-		SYS_IOCTL(data->fd, VIDIOC_S_CTRL, &ctrl);
+		v4l2_ioctl(data->fd, VIDIOC_S_CTRL, &ctrl);
 	}
 	if (exposure != orig_exposure) {
-		ctrl.id = V4L2_CID_EXPOSURE;
+		ctrl.id = V4L2_CID_EXPOSURE_ABSOLUTE;
 		ctrl.value = exposure;
-		SYS_IOCTL(data->fd, VIDIOC_S_CTRL, &ctrl);
+		v4l2_ioctl(data->fd, VIDIOC_S_CTRL, &ctrl);
 	}
 
 	return 0;
 }
 
 struct v4lprocessing_filter autogain_filter = {
-	autogain_active, autogain_calculate_lookup_tables
+	autogain_active, autogain_calculate_lookup_tables_exp
 };
+
diff --git a/lib/libv4lconvert/processing/libv4lprocessing-priv.h b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
index e4a29dd..43badef 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing-priv.h
+++ b/lib/libv4lconvert/processing/libv4lprocessing-priv.h
@@ -24,7 +24,9 @@
 #include "../control/libv4lcontrol.h"
 #include "../libv4lsyscall-priv.h"
 
-#define V4L2PROCESSING_UPDATE_RATE 10
+#define V4L2PROCESSING_UPDATE_RATE 1
+/* Size of lookup tables */
+#define LSIZE 1024
 
 struct v4lprocessing_data {
 	struct v4lcontrol_data *control;
@@ -32,15 +34,15 @@ struct v4lprocessing_data {
 	int do_process;
 	int controls_changed;
 	/* True if any of the lookup tables does not contain
-	   linear 0-255 */
+	   linear 0..LSIZE-1 */
 	int lookup_table_active;
 	/* Counts the number of processed frames until a
 	   V4L2PROCESSING_UPDATE_RATE overflow happens */
 	int lookup_table_update_counter;
 	/* RGB/BGR lookup tables */
-	unsigned char comp1[256];
-	unsigned char green[256];
-	unsigned char comp2[256];
+	unsigned short comp1[LSIZE];
+	unsigned short green[LSIZE];
+	unsigned short comp2[LSIZE];
 	/* Filter private data for filters which need it */
 	/* whitebalance.c data */
 	int green_avg;
@@ -48,7 +50,7 @@ struct v4lprocessing_data {
 	int comp2_avg;
 	/* gamma.c data */
 	int last_gamma;
-	unsigned char gamma_table[256];
+	unsigned char gamma_table[LSIZE];
 	/* autogain.c data */
 	int last_gain_correction;
 };
@@ -64,5 +66,6 @@ struct v4lprocessing_filter {
 extern struct v4lprocessing_filter whitebalance_filter;
 extern struct v4lprocessing_filter autogain_filter;
 extern struct v4lprocessing_filter gamma_filter;
+extern struct v4lprocessing_filter focus_filter;
 
 #endif
diff --git a/lib/libv4lconvert/processing/libv4lprocessing.c b/lib/libv4lconvert/processing/libv4lprocessing.c
index b061f50..b66c15a 100644
--- a/lib/libv4lconvert/processing/libv4lprocessing.c
+++ b/lib/libv4lconvert/processing/libv4lprocessing.c
@@ -31,6 +31,7 @@ static struct v4lprocessing_filter *filters[] = {
 	&whitebalance_filter,
 	&autogain_filter,
 	&gamma_filter,
+	&focus_filter,
 };
 
 struct v4lprocessing_data *v4lprocessing_create(int fd, struct v4lcontrol_data *control)
@@ -74,7 +75,7 @@ static void v4lprocessing_update_lookup_tables(struct v4lprocessing_data *data,
 {
 	int i;
 
-	for (i = 0; i < 256; i++) {
+	for (i = 0; i < LSIZE; i++) {
 		data->comp1[i] = i;
 		data->green[i] = i;
 		data->comp2[i] = i;
@@ -89,15 +90,34 @@ static void v4lprocessing_update_lookup_tables(struct v4lprocessing_data *data,
 	}
 }
 
-static void v4lprocessing_do_processing(struct v4lprocessing_data *data,
-		unsigned char *buf, const struct v4l2_format *fmt)
+static void v4lprocessing_do_processing_bayer10(struct v4lprocessing_data *data,
+		unsigned short *buf, const struct v4l2_format *fmt)
 {
 	int x, y;
+  		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+			for (x = 0; x < fmt->fmt.pix.width / 2; x++) {
+				*buf = data->green[*buf];
+				buf++;
+				*buf = data->comp1[*buf];
+				buf++;
+			}
+			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width * 2;
+			for (x = 0; x < fmt->fmt.pix.width / 2; x++) {
+				*buf = data->comp2[*buf];
+				buf++;
+				*buf = data->green[*buf];
+				buf++;
+			}
+			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width * 2;;
+		}
 
-	switch (fmt->fmt.pix.pixelformat) {
-	case V4L2_PIX_FMT_SGBRG8:
-	case V4L2_PIX_FMT_SGRBG8: /* Bayer patterns starting with green */
-		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
+}
+
+static void v4lprocessing_do_processing_bayer8(struct v4lprocessing_data *data,
+		unsigned char *buf, const struct v4l2_format *fmt)
+{
+	int x, y;
+  		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
 			for (x = 0; x < fmt->fmt.pix.width / 2; x++) {
 				*buf = data->green[*buf];
 				buf++;
@@ -113,8 +133,25 @@ static void v4lprocessing_do_processing(struct v4lprocessing_data *data,
 			}
 			buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width;
 		}
+
+}
+
+static void v4lprocessing_do_processing(struct v4lprocessing_data *data,
+		unsigned char *buf, const struct v4l2_format *fmt)
+{
+	int x, y;
+
+	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10: /* Bayer patterns starting with green */
+	  	v4lprocessing_do_processing_bayer10(data, buf, fmt);
 		break;
 
+	case V4L2_PIX_FMT_SGBRG8:
+	case V4L2_PIX_FMT_SGRBG8: /* Bayer patterns starting with green */
+	  	v4lprocessing_do_processing_bayer8(data, buf, fmt);
+		break;
+
+
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8: /* Bayer patterns *NOT* starting with green */
 		for (y = 0; y < fmt->fmt.pix.height / 2; y++) {
@@ -164,15 +201,20 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SBGGR8:
 	case V4L2_PIX_FMT_SRGGB8:
+	case V4L2_PIX_FMT_SGBRG10:
+	case V4L2_PIX_FMT_SGRBG10:
+	case V4L2_PIX_FMT_SBGGR10:
+	case V4L2_PIX_FMT_SRGGB10:
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		break;
 	default:
-		return; /* Non supported pix format */
+		printf("Processing: unsupported pix format\n");
+		break; /* Non supported pix format */
 	}
 
 	if (data->controls_changed ||
-			data->lookup_table_update_counter == V4L2PROCESSING_UPDATE_RATE) {
+			data->lookup_table_update_counter >= V4L2PROCESSING_UPDATE_RATE) {
 		data->controls_changed = 0;
 		data->lookup_table_update_counter = 0;
 		/* Do this after resetting lookup_table_update_counter so that filters can
@@ -181,8 +223,9 @@ void v4lprocessing_processing(struct v4lprocessing_data *data,
 	} else
 		data->lookup_table_update_counter++;
 
-	if (data->lookup_table_active)
+	if (data->lookup_table_active) {
 		v4lprocessing_do_processing(data, buf, fmt);
+	}
 
 	data->do_process = 0;
 }
diff --git a/lib/libv4lconvert/processing/whitebalance.c b/lib/libv4lconvert/processing/whitebalance.c
index c74069a..95eb9c7 100644
--- a/lib/libv4lconvert/processing/whitebalance.c
+++ b/lib/libv4lconvert/processing/whitebalance.c
@@ -27,8 +27,8 @@
 #include "libv4lprocessing-priv.h"
 #include "../libv4lconvert-priv.h" /* for PIX_FMT defines */
 
-#define CLIP256(color) (((color) > 0xff) ? 0xff : (((color) < 0) ? 0 : (color)))
 #define CLIP(color, min, max) (((color) > (max)) ? (max) : (((color) < (min)) ? (min) : (color)))
+#define CLIPLSIZE(color) CLIP(color, 0, LSIZE-1)
 
 static int whitebalance_active(struct v4lprocessing_data *data)
 {
@@ -43,6 +43,7 @@ static int whitebalance_active(struct v4lprocessing_data *data)
 	return wb;
 }
 
+/* Updates data->comp1/comp2 tables, that are later used by v4lprocessing_do_processing() */
 static int whitebalance_calculate_lookup_tables_generic(
 		struct v4lprocessing_data *data, int green_avg, int comp1_avg, int comp2_avg)
 {
@@ -51,12 +52,12 @@ static int whitebalance_calculate_lookup_tables_generic(
 	const int max_step = 128;
 
 	/* Clip averages (restricts maximum white balance correction) */
-	green_avg = CLIP(green_avg, 512, 3072);
-	comp1_avg = CLIP(comp1_avg, 512, 3072);
-	comp2_avg = CLIP(comp2_avg, 512, 3072);
+	green_avg = CLIP(green_avg, LSIZE*2, LSIZE*12);
+	comp1_avg = CLIP(comp1_avg, LSIZE*2, LSIZE*12);
+	comp2_avg = CLIP(comp2_avg, LSIZE*2, LSIZE*12);
 
 	/* First frame ? */
-	if (data->green_avg == 0) {
+	if (1 || data->green_avg == 0) {
 		data->green_avg = green_avg;
 		data->comp1_avg = comp1_avg;
 		data->comp2_avg = comp2_avg;
@@ -111,15 +112,54 @@ static int whitebalance_calculate_lookup_tables_generic(
 
 	avg_avg = (data->green_avg + data->comp1_avg + data->comp2_avg) / 3;
 
-	for (i = 0; i < 256; i++) {
-		data->comp1[i] = CLIP256(data->comp1[i] * avg_avg / data->comp1_avg);
-		data->green[i] = CLIP256(data->green[i] * avg_avg / data->green_avg);
-		data->comp2[i] = CLIP256(data->comp2[i] * avg_avg / data->comp2_avg);
+	for (i = 0; i < LSIZE; i++) {
+		data->comp1[i] = CLIPLSIZE(data->comp1[i] * avg_avg / data->comp1_avg);
+		data->green[i] = CLIPLSIZE(data->green[i] * avg_avg / data->green_avg);
+		data->comp2[i] = CLIPLSIZE(data->comp2[i] * avg_avg / data->comp2_avg);
 	}
 
 	return 1;
 }
 
+static int whitebalance_calculate_lookup_tables_bayer10(
+		struct v4lprocessing_data *data, unsigned short *buf,
+		const struct v4l2_format *fmt, int starts_with_green)
+{
+	int x, y, a1 = 0, a2 = 0, b1 = 0, b2 = 0;
+	int green_avg, comp1_avg, comp2_avg;
+
+	for (y = 0; y < fmt->fmt.pix.height; y += 2) {
+		for (x = 0; x < fmt->fmt.pix.width; x += 2) {
+			a1 += *buf++;
+			a2 += *buf++;
+		}
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width * 2;
+		for (x = 0; x < fmt->fmt.pix.width; x += 2) {
+			b1 += *buf++;
+			b2 += *buf++;
+		}
+		buf += fmt->fmt.pix.bytesperline - fmt->fmt.pix.width * 2;
+	}
+
+	if (starts_with_green) {
+		green_avg = a1 / 2 + b2 / 2;
+		comp1_avg = a2;
+		comp2_avg = b1;
+	} else {
+		green_avg = a2 / 2 + b1 / 2;
+		comp1_avg = a1;
+		comp2_avg = b2;
+	}
+
+	/* Norm avg to ~ 0 - 4095 */
+	green_avg /= fmt->fmt.pix.width * fmt->fmt.pix.height / 64;
+	comp1_avg /= fmt->fmt.pix.width * fmt->fmt.pix.height / 64;
+	comp2_avg /= fmt->fmt.pix.width * fmt->fmt.pix.height / 64;
+
+	return whitebalance_calculate_lookup_tables_generic(data, green_avg,
+			comp1_avg, comp2_avg);
+}
+
 static int whitebalance_calculate_lookup_tables_bayer(
 		struct v4lprocessing_data *data, unsigned char *buf,
 		const struct v4l2_format *fmt, int starts_with_green)
@@ -189,6 +229,9 @@ static int whitebalance_calculate_lookup_tables(
 		unsigned char *buf, const struct v4l2_format *fmt)
 {
 	switch (fmt->fmt.pix.pixelformat) {
+	case V4L2_PIX_FMT_SGRBG10: /* Bayer patterns starting with green */
+		return whitebalance_calculate_lookup_tables_bayer10(data, (void *) buf, fmt, 1);
+
 	case V4L2_PIX_FMT_SGBRG8:
 	case V4L2_PIX_FMT_SGRBG8: /* Bayer patterns starting with green */
 		return whitebalance_calculate_lookup_tables_bayer(data, buf, fmt, 1);
@@ -200,6 +243,9 @@ static int whitebalance_calculate_lookup_tables(
 	case V4L2_PIX_FMT_RGB24:
 	case V4L2_PIX_FMT_BGR24:
 		return whitebalance_calculate_lookup_tables_rgb(data, buf, fmt);
+
+	default:
+	        printf("Whitebalance: unknown format\n");
 	}
 
 	return 0; /* Should never happen */


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Camera support, Prague next week, sdlcam

[Hans Verkuil]

On 22/10/17 00:00, Pavel Machek wrote:
> Hi!
> 
> I'd still like to get some reasonable support for cellphone camera in
> Linux.
> 
> IMO first reasonable step is to merge sdlcam, then we can implement
> autofocus, improve autogain... and rest of the boring stuff. Ouch and
> media graph support would be nice. Currently, _nothing_ works with
> media graph device, such as N900.

Can you post your latest rebased patch for sdlcam for v4l-utils?

I'll do a review and will likely merge it for you. Yes, I've changed my
mind on that.

> 
> I'll talk about the issues at ELCE in few days:
> 
> https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> 
> Will someone else be there? Is there some place where v4l people meet?

Why don't we discuss this Tuesday morning at 9am? I have no interest in the
keynotes on that day, so those who are interested can get together.

I'll be at your presentation tomorrow and we can discuss a bit during
the following coffee break if time permits.

Regards,

	Hans

Camera support, Prague next week, sdlcam

[Hans Verkuil]

On 22/10/17 00:00, Pavel Machek wrote:
> Hi!
> 
> I'd still like to get some reasonable support for cellphone camera in
> Linux.
> 
> IMO first reasonable step is to merge sdlcam, then we can implement
> autofocus, improve autogain... and rest of the boring stuff. Ouch and
> media graph support would be nice. Currently, _nothing_ works with
> media graph device, such as N900.

Can you post your latest rebased patch for sdlcam for v4l-utils?

I'll do a review and will likely merge it for you. Yes, I've changed my
mind on that.

> 
> I'll talk about the issues at ELCE in few days:
> 
> https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> 
> Will someone else be there? Is there some place where v4l people meet?

Why don't we discuss this Tuesday morning at 9am? I have no interest in the
keynotes on that day, so those who are interested can get together.

I'll be at your presentation tomorrow and we can discuss a bit during
the following coffee break if time permits.

Regards,

	Hans

Re: Camera support, Prague next week, sdlcam

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1312 bytes --]

Hi!

> > I'd still like to get some reasonable support for cellphone camera in
> > Linux.
> > 
> > IMO first reasonable step is to merge sdlcam, then we can implement
> > autofocus, improve autogain... and rest of the boring stuff. Ouch and
> > media graph support would be nice. Currently, _nothing_ works with
> > media graph device, such as N900.
> 
> Can you post your latest rebased patch for sdlcam for v4l-utils?
> 
> I'll do a review and will likely merge it for you. Yes, I've changed my
> mind on that.

Ok, will do, thanks!

> > I'll talk about the issues at ELCE in few days:
> > 
> > https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> > 
> > Will someone else be there? Is there some place where v4l people meet?
> 
> Why don't we discuss this Tuesday morning at 9am? I have no interest in the
> keynotes on that day, so those who are interested can get together.
> 
> I'll be at your presentation tomorrow and we can discuss a bit during
> the following coffee break if time permits.

Ok, sounds like a plan. Lets confirm Tuesday morning tommorow.

Thanks,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Camera support, Prague next week, sdlcam

[Pavel Machek]

Hi!

> > I'd still like to get some reasonable support for cellphone camera in
> > Linux.
> > 
> > IMO first reasonable step is to merge sdlcam, then we can implement
> > autofocus, improve autogain... and rest of the boring stuff. Ouch and
> > media graph support would be nice. Currently, _nothing_ works with
> > media graph device, such as N900.
> 
> Can you post your latest rebased patch for sdlcam for v4l-utils?
> 
> I'll do a review and will likely merge it for you. Yes, I've changed my
> mind on that.

Ok, will do, thanks!

> > I'll talk about the issues at ELCE in few days:
> > 
> > https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> > 
> > Will someone else be there? Is there some place where v4l people meet?
> 
> Why don't we discuss this Tuesday morning at 9am? I have no interest in the
> keynotes on that day, so those who are interested can get together.
> 
> I'll be at your presentation tomorrow and we can discuss a bit during
> the following coffee break if time permits.

Ok, sounds like a plan. Lets confirm Tuesday morning tommorow.

Thanks,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20171022/955955a1/attachment.sig>

Re: Camera support, Prague next week, sdlcam

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

Hi!

> > I'll talk about the issues at ELCE in few days:
> > 
> > https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> > 
> > Will someone else be there? Is there some place where v4l people meet?
> 
> Why don't we discuss this Tuesday morning at 9am? I have no interest in the
> keynotes on that day, so those who are interested can get together.

Ok, what about 9:30am? The place near the elevators where we were
talking last time?

And.... sorry for late reply.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Camera support, Prague next week, sdlcam

[Pavel Machek]

Hi!

> > I'll talk about the issues at ELCE in few days:
> > 
> > https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
> > 
> > Will someone else be there? Is there some place where v4l people meet?
> 
> Why don't we discuss this Tuesday morning at 9am? I have no interest in the
> keynotes on that day, so those who are interested can get together.

Ok, what about 9:30am? The place near the elevators where we were
talking last time?

And.... sorry for late reply.
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20171023/00ab004f/attachment.sig>

Re: Camera support, Prague next week, sdlcam

[Hans Verkuil]

Sounds good. That's the elevators on level LL by the way.

Regards,

Hans

On October 23, 2017 8:54:49 PM GMT+02:00, Pavel Machek <pavel@ucw.cz> wrote:
>Hi!
>
>> > I'll talk about the issues at ELCE in few days:
>> > 
>> >
>https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
>> > 
>> > Will someone else be there? Is there some place where v4l people
>meet?
>> 
>> Why don't we discuss this Tuesday morning at 9am? I have no interest
>in the
>> keynotes on that day, so those who are interested can get together.
>
>Ok, what about 9:30am? The place near the elevators where we were
>talking last time?
>
>And.... sorry for late reply.
>									Pavel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Camera support, Prague next week, sdlcam

[Hans Verkuil]

Sounds good. That's the elevators on level LL by the way.

Regards,

Hans

On October 23, 2017 8:54:49 PM GMT+02:00, Pavel Machek <pavel@ucw.cz> wrote:
>Hi!
>
>> > I'll talk about the issues at ELCE in few days:
>> > 
>> >
>https://osseu17.sched.com/event/ByYH/cheap-complex-cameras-pavel-machek-denx-software-engineering-gmbh
>> > 
>> > Will someone else be there? Is there some place where v4l people
>meet?
>> 
>> Why don't we discuss this Tuesday morning at 9am? I have no interest
>in the
>> keynotes on that day, so those who are interested can get together.
>
>Ok, what about 9:30am? The place near the elevators where we were
>talking last time?
>
>And.... sorry for late reply.
>									Pavel

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Camera support, Prague next week, sdlcam

[Sakari Ailus]

On Mon, Oct 23, 2017 at 09:24:24PM +0200, Hans Verkuil wrote:
> Sounds good. That's the elevators on level LL by the way.

I'll be there, too!

-- 
Sakari Ailus
e-mail: sakari.ailus@iki.fi

Camera support, Prague next week, sdlcam

[Sakari Ailus]

On Mon, Oct 23, 2017 at 09:24:24PM +0200, Hans Verkuil wrote:
> Sounds good. That's the elevators on level LL by the way.

I'll be there, too!

-- 
Sakari Ailus
e-mail: sakari.ailus at iki.fi

Re: Camera support, Prague next week, sdlcam

[Mauro Carvalho Chehab]

Em Mon, 23 Oct 2017 23:15:57 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> On Mon, Oct 23, 2017 at 09:24:24PM +0200, Hans Verkuil wrote:
> > Sounds good. That's the elevators on level LL by the way.  
> 
> I'll be there, too!
> 

I'll try to join you there too.


Cheers,
Mauro

Camera support, Prague next week, sdlcam

[Mauro Carvalho Chehab]

Em Mon, 23 Oct 2017 23:15:57 +0300
Sakari Ailus <sakari.ailus@iki.fi> escreveu:

> On Mon, Oct 23, 2017 at 09:24:24PM +0200, Hans Verkuil wrote:
> > Sounds good. That's the elevators on level LL by the way.  
> 
> I'll be there, too!
> 

I'll try to join you there too.


Cheers,
Mauro

Nokia N9: fun with camera

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2484 bytes --]

Hi!

Sakari, I am actually playing with N9 camera, not N950. That comes
next.

And the clock error I mentioned ... seems to be
-EPROBE_DEFER. So... not an issue.

Strange thing is, that my sensors seems to have different resolution
from yours:

- entity 89: smiapp pixel_array 1-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
	                  device node name /dev/v4l-subdev9
		          pad0: Source [fmt:SRGGB10_1X10/3572x2464 field:none
	                      crop.bounds:(0,0)/3572x2464 crop:(0,0)/3572x2464]
 ->
             "smiapp binner 1-0010":0 [ENABLED,IMMUTABLE]

(And you mentioned width 3600, and SGRBG10).

I updated my scripts accordingly. Now I get

pavel@n900:~/g/tui/camera$ /my/tui/yavta/yavta -c5 -f SRGGB10
-F/tmp/foo -s 3572x2464 /dev/video1
Device /dev/video1 opened.
Device `OMAP3 ISP CSI2a output' on `media' is a video capture (without
mplanes) device.
Video format set: SRGGB10 (30314752) 3572x2464 (stride 7144) field
none buffer size 17602816
Video format: SRGGB10 (30314752) 3572x2464 (stride 7144) field none
buffer size 17602816
2 buffers requested.
length: 17602816 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb5cb1000.
length: 17602816 offset: 17604608 timestamp type/source: mono/EoF
Buffer 1/0 mapped at address 0xb4be7000.

...but here it hangs. (Kernel v4.13).

dmesg says:

[ 2862.229736] smiapp 1-0010: flip 0
[ 2862.229766] smiapp 1-0010: new pixel order RGGB
[ 2862.233764] smiapp 1-0010: 0x00021700 "min_frame_length_lines_bin"
= 166, 0xa6
[ 2862.234558] smiapp 1-0010: 0x00021702 "max_frame_length_lines_bin"
= 65535, 0xffff
[ 2862.235351] smiapp 1-0010: 0x00021704 "min_line_length_pck_bin" =
3812, 0xee4
[ 2862.236114] smiapp 1-0010: 0x00021706 "max_line_length_pck_bin" =
32752, 0x7ff0
[ 2862.238067] smiapp 1-0010: 0x00021708 "min_line_blanking_pck_bin" =
240, 0xf0
[ 2862.240844] smiapp 1-0010: 0x0002170a
"fine_integration_time_min_bin" = 0, 0x0
[ 2862.242553] smiapp 1-0010: 0x0002170c
"fine_integration_time_max_margin_bin" = 0, 0x0
[ 2862.242614] smiapp 1-0010: vblank   	 26
[ 2862.242645] smiapp 1-0010: hblank		240
[ 2862.242675] smiapp 1-0010: real timeperframe	100/839

I did some more experiments, but could not grab a frame.

Best regards,
									Pavel


-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: Nokia N9: fun with camera

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]

Hi!

> Sakari, I am actually playing with N9 camera, not N950. That comes
> next.
> 
> And the clock error I mentioned ... seems to be
> -EPROBE_DEFER. So... not an issue.

Hmm, and with similar config, I got N950 to work. ... which should
give me enough clues to get N9 to work. I guess I forgot to reset the
pipeline between the tries, or something.

For the record, this got me some data on n950:

 m.media_ctl( [ '-f', '"OMAP3 ISP CSI2a":0 [fmt:%s/%dx%d]' % (m.fmt, m.cap_x, m.cap_y) ] )
 m.media_ctl( [ '-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]' ] )

 # WORKS!!!!
 # pavel@n900:~/g/tui/camera$ sudo /my/tui/yavta/yavta
 # --capture=8 --skip 0 --format SGRBG10 --size 4272x3016 /dev/video1 --file=/tmp/delme#

...ouch. It only worked twice :-(. Either driver gets confused by my
attempts, or it relied on some other initialization code. Strange.

Best regards,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Nokia N9: fun with camera

[Pavel Machek]

Hi!

> Sakari, I am actually playing with N9 camera, not N950. That comes
> next.
> 
> And the clock error I mentioned ... seems to be
> -EPROBE_DEFER. So... not an issue.

Hmm, and with similar config, I got N950 to work. ... which should
give me enough clues to get N9 to work. I guess I forgot to reset the
pipeline between the tries, or something.

For the record, this got me some data on n950:

 m.media_ctl( [ '-f', '"OMAP3 ISP CSI2a":0 [fmt:%s/%dx%d]' % (m.fmt, m.cap_x, m.cap_y) ] )
 m.media_ctl( [ '-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]' ] )

 # WORKS!!!!
 # pavel at n900:~/g/tui/camera$ sudo /my/tui/yavta/yavta
 # --capture=8 --skip 0 --format SGRBG10 --size 4272x3016 /dev/video1 --file=/tmp/delme#

...ouch. It only worked twice :-(. Either driver gets confused by my
attempts, or it relied on some other initialization code. Strange.

Best regards,
									Pavel

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20171101/6e31e864/attachment.sig>

Re: Nokia N9: fun with camera

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 2629 bytes --]

Hi!

> > Sakari, I am actually playing with N9 camera, not N950. That comes
> > next.
> > 
> > And the clock error I mentioned ... seems to be
> > -EPROBE_DEFER. So... not an issue.
> 
> Hmm, and with similar config, I got N950 to work. ... which should
> give me enough clues to get N9 to work. I guess I forgot to reset the
> pipeline between the tries, or something.
> 
> For the record, this got me some data on n950:
> 
>  m.media_ctl( [ '-f', '"OMAP3 ISP CSI2a":0 [fmt:%s/%dx%d]' % (m.fmt, m.cap_x, m.cap_y) ] )
>  m.media_ctl( [ '-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]' ] )
> 
>  # WORKS!!!!
>  # pavel@n900:~/g/tui/camera$ sudo /my/tui/yavta/yavta
>  # --capture=8 --skip 0 --format SGRBG10 --size 4272x3016 /dev/video1 --file=/tmp/delme#
> 
> ...ouch. It only worked twice :-(. Either driver gets confused by my
> attempts, or it relied on some other initialization code. Strange.

Hmm, so it works "reliably" after boot. But it also locks up machine
with high probability. If you have N950, it might still be handy...

Messages are:

['-r']
['-f', '"OMAP3 ISP CSI2a":0 [fmt:SGRBG10/4272x3016]']
Warning: the -f option is deprecated and has been replaced by -V.
['-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]']
Testing: is raw
Testing:  /my/tui/yavta/yavta --capture=8 --skip 0 --format SGRBG10
--size 4272x3016 /dev/video_sensor --file=/tmp/delme#
Error opening device /dev/video_sensor: Permission denied (13).
Raw mode is  1
Raw mode is  1
Testing: is raw
Testing:  /my/tui/yavta/yavta --capture=8 --skip 0 --format SGRBG10
--size 4272x3016 /dev/video_sensor --file=/tmp/delme#
Device /dev/video_sensor opened.
Device `OMAP3 ISP CSI2a output' on `media' is a video capture (without
mplanes) device.
Video format set: SGRBG10 (30314142) 4272x3016 (stride 8544) field
none buffer size 25768704
Video format: SGRBG10 (30314142) 4272x3016 (stride 8544) field none
buffer size 25768704
1 buffers requested.
length: 25768704 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb54e9000.
0 (0) [-] none 0 25768704 B 691.756907 691.758403 6.212 fps ts
mono/EoF
1 (0) [-] none 1 25768704 B 694.821025 694.821422 0.326 fps ts
mono/EoF
2 (0) [E] none 2 25768704 B 698.530833 698.534739 0.270 fps ts
mono/EoF
3 (0) [-] none 3 25768704 B 700.788127 700.790996 0.443 fps ts
mono/EoF
(and here it locked up :-(. Sometimes it captures more.)

Thanks,

								Pavel




-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Nokia N9: fun with camera

[Pavel Machek]

Hi!

> > Sakari, I am actually playing with N9 camera, not N950. That comes
> > next.
> > 
> > And the clock error I mentioned ... seems to be
> > -EPROBE_DEFER. So... not an issue.
> 
> Hmm, and with similar config, I got N950 to work. ... which should
> give me enough clues to get N9 to work. I guess I forgot to reset the
> pipeline between the tries, or something.
> 
> For the record, this got me some data on n950:
> 
>  m.media_ctl( [ '-f', '"OMAP3 ISP CSI2a":0 [fmt:%s/%dx%d]' % (m.fmt, m.cap_x, m.cap_y) ] )
>  m.media_ctl( [ '-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]' ] )
> 
>  # WORKS!!!!
>  # pavel at n900:~/g/tui/camera$ sudo /my/tui/yavta/yavta
>  # --capture=8 --skip 0 --format SGRBG10 --size 4272x3016 /dev/video1 --file=/tmp/delme#
> 
> ...ouch. It only worked twice :-(. Either driver gets confused by my
> attempts, or it relied on some other initialization code. Strange.

Hmm, so it works "reliably" after boot. But it also locks up machine
with high probability. If you have N950, it might still be handy...

Messages are:

['-r']
['-f', '"OMAP3 ISP CSI2a":0 [fmt:SGRBG10/4272x3016]']
Warning: the -f option is deprecated and has been replaced by -V.
['-l', '"OMAP3 ISP CSI2a":1 -> "OMAP3 ISP CSI2a output":0[1]']
Testing: is raw
Testing:  /my/tui/yavta/yavta --capture=8 --skip 0 --format SGRBG10
--size 4272x3016 /dev/video_sensor --file=/tmp/delme#
Error opening device /dev/video_sensor: Permission denied (13).
Raw mode is  1
Raw mode is  1
Testing: is raw
Testing:  /my/tui/yavta/yavta --capture=8 --skip 0 --format SGRBG10
--size 4272x3016 /dev/video_sensor --file=/tmp/delme#
Device /dev/video_sensor opened.
Device `OMAP3 ISP CSI2a output' on `media' is a video capture (without
mplanes) device.
Video format set: SGRBG10 (30314142) 4272x3016 (stride 8544) field
none buffer size 25768704
Video format: SGRBG10 (30314142) 4272x3016 (stride 8544) field none
buffer size 25768704
1 buffers requested.
length: 25768704 offset: 0 timestamp type/source: mono/EoF
Buffer 0/0 mapped at address 0xb54e9000.
0 (0) [-] none 0 25768704 B 691.756907 691.758403 6.212 fps ts
mono/EoF
1 (0) [-] none 1 25768704 B 694.821025 694.821422 0.326 fps ts
mono/EoF
2 (0) [E] none 2 25768704 B 698.530833 698.534739 0.270 fps ts
mono/EoF
3 (0) [-] none 3 25768704 B 700.788127 700.790996 0.443 fps ts
mono/EoF
(and here it locked up :-(. Sometimes it captures more.)

Thanks,

								Pavel




-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20171101/18dc9d80/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 945 bytes --]

Hi!

> Please don't add a new application under lib/. It is fine if you want
> some testing application, if the ones there aren't enough, but please
> place it under contrib/test/.
> 
> You should likely take a look at v4l2grab first, as it could have
> almost everything you would need.

I really need some kind of video output. v4l2grab is not useful
there. v4l2gl might be, but I don't think I have enough dependencies.

Umm, and it looks like libv4l can not automatically convert from
GRBG10.. and if it could, going through RGB24 would probably be too
slow on this device :-(.

> IMO, the above belongs to a separate processing module under
> 	lib/libv4lconvert/processing/

Is there an example using autogain/autowhitebalance from
libv4lconvert?

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> Please don't add a new application under lib/. It is fine if you want
> some testing application, if the ones there aren't enough, but please
> place it under contrib/test/.
> 
> You should likely take a look at v4l2grab first, as it could have
> almost everything you would need.

I really need some kind of video output. v4l2grab is not useful
there. v4l2gl might be, but I don't think I have enough dependencies.

Umm, and it looks like libv4l can not automatically convert from
GRBG10.. and if it could, going through RGB24 would probably be too
slow on this device :-(.

> IMO, the above belongs to a separate processing module under
> 	lib/libv4lconvert/processing/

Is there an example using autogain/autowhitebalance from
libv4lconvert?

Best regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/c41537cc/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Tue, 25 Apr 2017 00:07:01 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > Please don't add a new application under lib/. It is fine if you want
> > some testing application, if the ones there aren't enough, but please
> > place it under contrib/test/.
> > 
> > You should likely take a look at v4l2grab first, as it could have
> > almost everything you would need.  
> 
> I really need some kind of video output. v4l2grab is not useful
> there. v4l2gl might be, but I don't think I have enough dependencies.

Well, you could use some app to show the snaps that v4l2grab takes.

Yeah, compiling v4l2gl on N9 can indeed be complex. I suspect that it 
shouldn't hard to compile xawtv there (probably disabling some optional
features).

> Umm, and it looks like libv4l can not automatically convert from
> GRBG10.. and if it could, going through RGB24 would probably be too
> slow on this device :-(.

I suspect it shouldn't be hard to add support for GRBG10. It already
supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
(to both RGB and YUV formats).

How it would preform is another question ;)

> > IMO, the above belongs to a separate processing module under
> > 	lib/libv4lconvert/processing/  
> 
> Is there an example using autogain/autowhitebalance from
> libv4lconvert?

Well, if you plug a USB camera without those controls, it should
automatically expose controls for it, as if the device had such
controls.

Thanks,
Mauro

support autofocus / autogain in libv4l2

[Mauro Carvalho Chehab]

Em Tue, 25 Apr 2017 00:07:01 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > Please don't add a new application under lib/. It is fine if you want
> > some testing application, if the ones there aren't enough, but please
> > place it under contrib/test/.
> > 
> > You should likely take a look at v4l2grab first, as it could have
> > almost everything you would need.  
> 
> I really need some kind of video output. v4l2grab is not useful
> there. v4l2gl might be, but I don't think I have enough dependencies.

Well, you could use some app to show the snaps that v4l2grab takes.

Yeah, compiling v4l2gl on N9 can indeed be complex. I suspect that it 
shouldn't hard to compile xawtv there (probably disabling some optional
features).

> Umm, and it looks like libv4l can not automatically convert from
> GRBG10.. and if it could, going through RGB24 would probably be too
> slow on this device :-(.

I suspect it shouldn't be hard to add support for GRBG10. It already
supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
(to both RGB and YUV formats).

How it would preform is another question ;)

> > IMO, the above belongs to a separate processing module under
> > 	lib/libv4lconvert/processing/  
> 
> Is there an example using autogain/autowhitebalance from
> libv4lconvert?

Well, if you plug a USB camera without those controls, it should
automatically expose controls for it, as if the device had such
controls.

Thanks,
Mauro

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1919 bytes --]

Hi!

> > > Please don't add a new application under lib/. It is fine if you want
> > > some testing application, if the ones there aren't enough, but please
> > > place it under contrib/test/.
> > > 
> > > You should likely take a look at v4l2grab first, as it could have
> > > almost everything you would need.  
> > 
> > I really need some kind of video output. v4l2grab is not useful
> > there. v4l2gl might be, but I don't think I have enough dependencies.
> 
> Well, you could use some app to show the snaps that v4l2grab takes.

That would be too slow :-(.

> Yeah, compiling v4l2gl on N9 can indeed be complex. I suspect that it 
> shouldn't hard to compile xawtv there (probably disabling some optional
> features).

I do have mplayer working, but that one is not linked against libv4l2
:-(.

> > Umm, and it looks like libv4l can not automatically convert from
> > GRBG10.. and if it could, going through RGB24 would probably be too
> > slow on this device :-(.
> 
> I suspect it shouldn't be hard to add support for GRBG10. It already
> supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
> (to both RGB and YUV formats).

Is 16bit bayer a recent development? I can't see it in

commit 374806e868f5a7a48ecffde4c6a1abfcfa5ccd65
Author: Hans Verkuil <hans.verkuil@cisco.com>
Date:   Fri Apr 22 09:31:57 2016 +0200

> > Is there an example using autogain/autowhitebalance from
> > libv4lconvert?
> 
> Well, if you plug a USB camera without those controls, it should
> automatically expose controls for it, as if the device had such
> controls.

And settings are persistent, so I can enable autogain, then lauch
something like xawtv, and it will automatically get autogain? Ok,
good.

Regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > > Please don't add a new application under lib/. It is fine if you want
> > > some testing application, if the ones there aren't enough, but please
> > > place it under contrib/test/.
> > > 
> > > You should likely take a look at v4l2grab first, as it could have
> > > almost everything you would need.  
> > 
> > I really need some kind of video output. v4l2grab is not useful
> > there. v4l2gl might be, but I don't think I have enough dependencies.
> 
> Well, you could use some app to show the snaps that v4l2grab takes.

That would be too slow :-(.

> Yeah, compiling v4l2gl on N9 can indeed be complex. I suspect that it 
> shouldn't hard to compile xawtv there (probably disabling some optional
> features).

I do have mplayer working, but that one is not linked against libv4l2
:-(.

> > Umm, and it looks like libv4l can not automatically convert from
> > GRBG10.. and if it could, going through RGB24 would probably be too
> > slow on this device :-(.
> 
> I suspect it shouldn't be hard to add support for GRBG10. It already
> supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
> (to both RGB and YUV formats).

Is 16bit bayer a recent development? I can't see it in

commit 374806e868f5a7a48ecffde4c6a1abfcfa5ccd65
Author: Hans Verkuil <hans.verkuil@cisco.com>
Date:   Fri Apr 22 09:31:57 2016 +0200

> > Is there an example using autogain/autowhitebalance from
> > libv4lconvert?
> 
> Well, if you plug a USB camera without those controls, it should
> automatically expose controls for it, as if the device had such
> controls.

And settings are persistent, so I can enable autogain, then lauch
something like xawtv, and it will automatically get autogain? Ok,
good.

Regards,
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/23ad62a6/attachment.sig>

Re: support autofocus / autogain in libv4l2

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3230 bytes --]

Hi!

> > Umm, and it looks like libv4l can not automatically convert from
> > GRBG10.. and if it could, going through RGB24 would probably be too
> > slow on this device :-(.
> 
> I suspect it shouldn't be hard to add support for GRBG10. It already
> supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
> (to both RGB and YUV formats).

Proper format for 16 bit bayer would be tricky, AFAICT. Anyway, does
this look reasonable? It does not work too well here, since omap3isp
driver does not seem to support ENUM_FMT. (And I get just half of the
vertical image, strange. Interlacing?)

Best regards,
								Pavel

diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
index d3d8936..2a469b2 100644
--- a/lib/libv4lconvert/libv4lconvert.c
+++ b/lib/libv4lconvert/libv4lconvert.c
@@ -123,6 +126,8 @@ static const struct v4lconvert_pixfmt supported_src_pixfmts[] = {
 	{ V4L2_PIX_FMT_SGRBG8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_SRGGB8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_STV0680,		 8,	 8,	 8,	1 },
+
+	{ V4L2_PIX_FMT_SGRBG10,		16,	 8,	 8,	1 },
 	/* compressed bayer */
 	{ V4L2_PIX_FMT_SPCA561,		 0,	 9,	 9,	1 },
 	{ V4L2_PIX_FMT_SN9C10X,		 0,	 9,	 9,	1 },
@@ -668,6 +680,7 @@ static int v4lconvert_processing_needs_double_conversion(
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SRGGB8:
 	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10:
 		return 0;
 	}
 	switch (dest_pix_fmt) {
@@ -694,6 +707,17 @@ unsigned char *v4lconvert_alloc_buffer(int needed,
 	return *buf;
 }
 
+static void v4lconvert_10to8(void *_src, unsigned char *dst, int width, int height)
+{
+	int i;
+	uint16_t *src = _src;
+	
+	printf("sizes %d x %d\n", width, height);
+	for (i=0; i<width*height; i++) {
+		dst[i] = src[i] >> 2;
+	}
+}
+
 int v4lconvert_oom_error(struct v4lconvert_data *data)
 {
 	V4LCONVERT_ERR("could not allocate memory\n");
@@ -867,7 +893,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 #endif
 	case V4L2_PIX_FMT_SN9C2028:
 	case V4L2_PIX_FMT_SQ905C:
-	case V4L2_PIX_FMT_STV0680: { /* Not compressed but needs some shuffling */
+	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10: { /* Not compressed but needs some shuffling */
 		unsigned char *tmpbuf;
 		struct v4l2_format tmpfmt = *fmt;
 
@@ -877,6 +904,11 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			return v4lconvert_oom_error(data);
 
 		switch (src_pix_fmt) {
+		case V4L2_PIX_FMT_SGRBG10:
+			v4lconvert_10to8(src, tmpbuf, width, height);
+
+			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG8;
+			break;
 		case V4L2_PIX_FMT_SPCA561:
 			v4lconvert_decode_spca561(src, tmpbuf, width, height);
 			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGBRG8;
@@ -949,6 +981,7 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			V4LCONVERT_ERR("short raw bayer data frame\n");
 			errno = EPIPE;
 			result = -1;
+			/* FIXME: but then we proceed anyway?! */
 		}
 		switch (dest_pix_fmt) {
 		case V4L2_PIX_FMT_RGB24:




-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

support autofocus / autogain in libv4l2

[Pavel Machek]

Hi!

> > Umm, and it looks like libv4l can not automatically convert from
> > GRBG10.. and if it could, going through RGB24 would probably be too
> > slow on this device :-(.
> 
> I suspect it shouldn't be hard to add support for GRBG10. It already
> supports 8 and 16 bits Bayer formats, at lib/libv4lconvert/bayer.c
> (to both RGB and YUV formats).

Proper format for 16 bit bayer would be tricky, AFAICT. Anyway, does
this look reasonable? It does not work too well here, since omap3isp
driver does not seem to support ENUM_FMT. (And I get just half of the
vertical image, strange. Interlacing?)

Best regards,
								Pavel

diff --git a/lib/libv4lconvert/libv4lconvert.c b/lib/libv4lconvert/libv4lconvert.c
index d3d8936..2a469b2 100644
--- a/lib/libv4lconvert/libv4lconvert.c
+++ b/lib/libv4lconvert/libv4lconvert.c
@@ -123,6 +126,8 @@ static const struct v4lconvert_pixfmt supported_src_pixfmts[] = {
 	{ V4L2_PIX_FMT_SGRBG8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_SRGGB8,		 8,	 8,	 8,	1 },
 	{ V4L2_PIX_FMT_STV0680,		 8,	 8,	 8,	1 },
+
+	{ V4L2_PIX_FMT_SGRBG10,		16,	 8,	 8,	1 },
 	/* compressed bayer */
 	{ V4L2_PIX_FMT_SPCA561,		 0,	 9,	 9,	1 },
 	{ V4L2_PIX_FMT_SN9C10X,		 0,	 9,	 9,	1 },
@@ -668,6 +680,7 @@ static int v4lconvert_processing_needs_double_conversion(
 	case V4L2_PIX_FMT_SGRBG8:
 	case V4L2_PIX_FMT_SRGGB8:
 	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10:
 		return 0;
 	}
 	switch (dest_pix_fmt) {
@@ -694,6 +707,17 @@ unsigned char *v4lconvert_alloc_buffer(int needed,
 	return *buf;
 }
 
+static void v4lconvert_10to8(void *_src, unsigned char *dst, int width, int height)
+{
+	int i;
+	uint16_t *src = _src;
+	
+	printf("sizes %d x %d\n", width, height);
+	for (i=0; i<width*height; i++) {
+		dst[i] = src[i] >> 2;
+	}
+}
+
 int v4lconvert_oom_error(struct v4lconvert_data *data)
 {
 	V4LCONVERT_ERR("could not allocate memory\n");
@@ -867,7 +893,8 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 #endif
 	case V4L2_PIX_FMT_SN9C2028:
 	case V4L2_PIX_FMT_SQ905C:
-	case V4L2_PIX_FMT_STV0680: { /* Not compressed but needs some shuffling */
+	case V4L2_PIX_FMT_STV0680:
+	case V4L2_PIX_FMT_SGRBG10: { /* Not compressed but needs some shuffling */
 		unsigned char *tmpbuf;
 		struct v4l2_format tmpfmt = *fmt;
 
@@ -877,6 +904,11 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			return v4lconvert_oom_error(data);
 
 		switch (src_pix_fmt) {
+		case V4L2_PIX_FMT_SGRBG10:
+			v4lconvert_10to8(src, tmpbuf, width, height);
+
+			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGRBG8;
+			break;
 		case V4L2_PIX_FMT_SPCA561:
 			v4lconvert_decode_spca561(src, tmpbuf, width, height);
 			tmpfmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SGBRG8;
@@ -949,6 +981,7 @@ static int v4lconvert_convert_pixfmt(struct v4lconvert_data *data,
 			V4LCONVERT_ERR("short raw bayer data frame\n");
 			errno = EPIPE;
 			result = -1;
+			/* FIXME: but then we proceed anyway?! */
 		}
 		switch (dest_pix_fmt) {
 		case V4L2_PIX_FMT_RGB24:




-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20170425/a03b799f/attachment-0001.sig>

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 1144 bytes --]

Hi!

> Hmm... if the idea is to have a control that doesn't do ringing
> compensation, then it should be clear at the control's descriptions
> that:
> 
> - V4L2_CID_FOCUS_ABSOLUTE should be used if the VCM has ringing
>   compensation;
> - V4L2_CID_VOICE_COIL_CURRENT and V4L2_CID_VOICE_COIL_RING_COMPENSATION
>   should be used otherwise.
> 
> Btw, if the rationale for this patch is to support devices without
> ring compensation, so, both controls and their descriptions should
> be added at the same time, together with a patchset that would be
> using both.
> 
> > How about adding such an explanation added to the commit message?
> 
> It is not enough. Documentation should be clear that VCM devices
> with ring compensation should use V4L2_CID_FOCUS_ABSOLUTE.

Is ring compensation actually a big deal? We do not publish enough
information to userland about how fast the autofocus system is,
anyway, so it looks like userland can't depend on such details...
									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Mauro Carvalho Chehab]

Em Sat, 29 Apr 2017 00:00:05 +0200
Pavel Machek <pavel@ucw.cz> escreveu:

> Hi!
> 
> > Hmm... if the idea is to have a control that doesn't do ringing
> > compensation, then it should be clear at the control's descriptions
> > that:
> > 
> > - V4L2_CID_FOCUS_ABSOLUTE should be used if the VCM has ringing
> >   compensation;
> > - V4L2_CID_VOICE_COIL_CURRENT and V4L2_CID_VOICE_COIL_RING_COMPENSATION
> >   should be used otherwise.
> > 
> > Btw, if the rationale for this patch is to support devices without
> > ring compensation, so, both controls and their descriptions should
> > be added at the same time, together with a patchset that would be
> > using both.
> >   
> > > How about adding such an explanation added to the commit message?  
> > 
> > It is not enough. Documentation should be clear that VCM devices
> > with ring compensation should use V4L2_CID_FOCUS_ABSOLUTE.  
> 
> Is ring compensation actually a big deal? We do not publish enough
> information to userland about how fast the autofocus system is,
> anyway, so it looks like userland can't depend on such details...

Well, I guess a V4L2 event could be used to identify the VCM's
current position and/or notify when the movement finished.

Anyway, the point is:

If V4L2_CID_VOICE_COIL_CURRENT would do the same as:
	V4L2_CID_FOCUS_ABSOLUTE
or:
	max - V4L2_CID_FOCUS_ABSOLUTE

there's no reason to create a new control, as the existing control
was already created to control the VCM current [1].

[1] Ok, we need to better document it, but that's a separate issue

We should create a new control only if it is doing something
different than the "standard" way of controlling a Voice Coil Motor.

On such case, the difference between controlling VCM via
V4L2_CID_VOICE_COIL_CURRENT or via V4L2_CID_FOCUS_ABSOLUTE should be 
clearly stated, as we expect that the other devices with the same need 
will implement the same control set and the same max/min convention
(e. g. max integer value meaning closest focus, min integer value
meaning infinite).

Thanks,
Mauro

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Sakari Ailus]

Hi Mauro,

On Wed, Apr 19, 2017 at 10:51:18AM -0300, Mauro Carvalho Chehab wrote:
> Em Sun, 16 Apr 2017 12:12:10 +0300
> Sakari Ailus <sakari.ailus@iki.fi> escreveu:
> 
> > Hi Mauro,
> > 
> > On Fri, Apr 14, 2017 at 11:23:32PM -0300, Mauro Carvalho Chehab wrote:
> > > Hi Sakari,
> > > 
> > > Em Tue, 14 Feb 2017 14:20:22 +0200
> > > Sakari Ailus <sakari.ailus@linux.intel.com> escreveu:
> > >   
> > > > Add a V4L2 control class for voice coil lens driver devices. These are
> > > > simple devices that are used to move a camera lens from its resting
> > > > position.  
> > > 
> > > From some past threads with this patch, you mentioned that:
> > > 
> > > "The FOCUS_ABSOLUTE control really is not a best control ID to
> > >  control a voice coil driver's current."
> > > 
> > > However, I'm not seeing any explanation there at the thread, at
> > > the patch description or at the documentation explaining why, and,
> > > more important, when someone should use the focus control or the
> > > camera voice coil control.  
> > 
> > It should be available in the thread.
> 
> The email thread is not registered at git logs nor at the API spec.
> 
> > Nevertheless, V4L2_CID_FOCUS_ABSOLUTE
> > is documented as follows (emphasis mine):
> > 
> > 	This control sets the *focal point* of the camera to the specified
> > 	position. The unit is undefined. Positive values set the focus
> > 	closer to the camera, negative values towards infinity.
> > 
> > What you control in voice coil devices is current (in Ampères) and the
> > current only has a relatively loose relation to the focal point.
> 
> The real problem I'm seeing here is that this control is already
> used by voice coil motor (VCM). Several UVC-based Logitech cameras
> come with VCM, like their QuickCam Pro-series webcams:
> 
> 	https://secure.logitech.com/en-hk/articles/3231
> 
> The voice coil can be seen on this picture:
> 	https://photo.stackexchange.com/questions/48678/can-i-modify-a-logitech-c615-webcam-for-infinity-focus

There may be voice coil lens implementations that are indirectly controlled
through this control. Those are hardware solutions that have been taken in
UVC webcams, for instance. The UVC standard itself uses millimeters.

Lens systems based on voice coils generally cannot focus at a given exact
distance for they have no concept of focussing at a particular distance.
Instead, an auto focus algorithm analyses the image data (or statistics of
image data) to control the lens --- in other words, to set current, not
distance.

As the auto focus algorithms require both image data (or statistics) and
access to lens voice coil as well as for algorithmic complexity, they are
typically implemented in user space.

In other words, the VOICE_COIL_CURRENT control is thus used by user space to
implement what the user expects from FOCUS_AUTO control. It could be
implemented in libv4l2 or a different user space component.
VOICE_COIL_CURRENT control is not a control which is expected to be used by
an end user application --- unlike FOCUS_AUTO.

This camera module datasheet shows the dependency between current and lens
position. See "Performance Diagram" on page 16:

<URL:http://www.trulyamerica.com/wp-content/uploads/CM7945-B1200BA-E_V1.1.pdf>

As you can see, lens position may start changing by applying a current
between 14 and 34 mA, but exactly how much is specific to a given camera
module unit. This means that there is no direct mapping between the current
and the focus distance.

Ringing compensation is another matter. Voice coils do ring unless it is
compensated, something that the user of a control focussing at a particular
distance expects. See the diagram on page 13:

<URL:http://www.datasheetspdf.com/datasheet/download.php?id=840322>

Do note that even this is not uniform with systems with the dw9714 lens
controller: the properties depend on the lens spring spring constant and the
weight of the moving lens package, for instance.

For these reasons I do think that this warrants having a specific control
for such devices.

Additionally, there will be controls related to ringing compensation. The
user (for an auto focus algorithm) still might want to disable the hardware
ringing compensation so a menu control would be needed for the purpose.
That's something that can well be addressed later on, just FYI.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [PATCH v3 1/2] v4l: Add camera voice coil lens control class, current control

[Pavel Machek]

[-- Attachment #1: Type: text/plain, Size: 3534 bytes --]

Hi!

> > > Nevertheless, V4L2_CID_FOCUS_ABSOLUTE
> > > is documented as follows (emphasis mine):
> > > 
> > > 	This control sets the *focal point* of the camera to the specified
> > > 	position. The unit is undefined. Positive values set the focus
> > > 	closer to the camera, negative values towards infinity.
> > > 
> > > What you control in voice coil devices is current (in Ampères) and the
> > > current only has a relatively loose relation to the focal point.
> > 
> > The real problem I'm seeing here is that this control is already
> > used by voice coil motor (VCM). Several UVC-based Logitech cameras
> > come with VCM, like their QuickCam Pro-series webcams:
> > 
> > 	https://secure.logitech.com/en-hk/articles/3231
> > 
> > The voice coil can be seen on this picture:
> > 	https://photo.stackexchange.com/questions/48678/can-i-modify-a-logitech-c615-webcam-for-infinity-focus
> 
> There may be voice coil lens implementations that are indirectly controlled
> through this control. Those are hardware solutions that have been taken in
> UVC webcams, for instance. The UVC standard itself uses millimeters.
> 
> Lens systems based on voice coils generally cannot focus at a given exact
> distance for they have no concept of focussing at a particular distance.
> Instead, an auto focus algorithm analyses the image data (or statistics of
> image data) to control the lens --- in other words, to set current, not
> distance.

Well, you are right that voice coil does not focus on _exact_
distance. I guess nothing ever focuses on _exact_ distance ;-). (Ok,
voice coils may be worse then other systems.)

> As the auto focus algorithms require both image data (or statistics) and
> access to lens voice coil as well as for algorithmic complexity, they are
> typically implemented in user space.
> 
> In other words, the VOICE_COIL_CURRENT control is thus used by user space to
> implement what the user expects from FOCUS_AUTO control. It could be
> implemented in libv4l2 or a different user space component.
> VOICE_COIL_CURRENT control is not a control which is expected to be used by
> an end user application --- unlike FOCUS_AUTO.

End user application definitely _wants_ to control voice coil control
directly. Original Maemo application has landscape-mode, which
presumably just sets control to infinity. FCam application has full
manual focus. I'm writing SDLcam, it also has manual focus.

N900 camera has depth of field of 2 diopters. That means you can have
everything from 0.5m to infinity in focus... At normal range manual
focus makes a lot of sense. At macro range, yes, autofocus is needed.

Applications want to know focus in diopters... user wants to
know. Even when autofocus is active, it is good to know where the
camera is focused to monitor its progress.

> Additionally, there will be controls related to ringing compensation. The
> user (for an auto focus algorithm) still might want to disable the hardware
> ringing compensation so a menu control would be needed for the purpose.
> That's something that can well be addressed later on, just FYI.

Well, I don't see why we can't add ringing compensation controls to
the FOCUS_ABSOLUTE. Yes, userspace should know that it is the voice
coil. But userspace should be also able to get/set approximate focus
distance (diopters).

									Pavel
-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[PATCH v3 2/2] ad5820: Use VOICE_COIL_CURRENT control

[Sakari Ailus]

Add V4L2_CID_VOICE_COIL_CURRENT control support to the ad5820 driver. The
usage of the control is equivalent to how V4L2_CID_FOCUS_ABSOLUTE was used
by the driver. The old control remains supported.

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
---
 drivers/media/i2c/ad5820.c | 27 +++++++++++++++++++++------
 1 file changed, 21 insertions(+), 6 deletions(-)

diff --git a/drivers/media/i2c/ad5820.c b/drivers/media/i2c/ad5820.c
index a9026a91..e5ff1a2 100644
--- a/drivers/media/i2c/ad5820.c
+++ b/drivers/media/i2c/ad5820.c
@@ -51,7 +51,7 @@ struct ad5820_device {
 	struct regulator *vana;
 
 	struct v4l2_ctrl_handler ctrls;
-	u32 focus_absolute;
+	struct v4l2_ctrl *focus, *curr;
 	u32 focus_ramp_time;
 	u32 focus_ramp_mode;
 
@@ -59,6 +59,7 @@ struct ad5820_device {
 	int power_count;
 
 	bool standby;
+	bool in_set_ctrl;
 };
 
 static int ad5820_write(struct ad5820_device *coil, u16 data)
@@ -98,7 +99,7 @@ static int ad5820_update_hw(struct ad5820_device *coil)
 	status = RAMP_US_TO_CODE(coil->focus_ramp_time);
 	status |= coil->focus_ramp_mode
 		? AD5820_RAMP_MODE_64_16 : AD5820_RAMP_MODE_LINEAR;
-	status |= coil->focus_absolute << AD5820_DAC_SHIFT;
+	status |= coil->curr->val << AD5820_DAC_SHIFT;
 
 	if (coil->standby)
 		status |= AD5820_POWER_DOWN;
@@ -160,9 +161,16 @@ static int ad5820_set_ctrl(struct v4l2_ctrl *ctrl)
 	struct ad5820_device *coil =
 		container_of(ctrl->handler, struct ad5820_device, ctrls);
 
+	if (coil->in_set_ctrl)
+		return 0;
+
 	switch (ctrl->id) {
 	case V4L2_CID_FOCUS_ABSOLUTE:
-		coil->focus_absolute = ctrl->val;
+	case V4L2_CID_VOICE_COIL_CURRENT:
+		coil->in_set_ctrl = true;
+		__v4l2_ctrl_s_ctrl(ctrl == coil->focus ?
+				   coil->curr : coil->focus, ctrl->val);
+		coil->in_set_ctrl = false;
 		return ad5820_update_hw(coil);
 	}
 
@@ -189,14 +197,21 @@ static int ad5820_init_controls(struct ad5820_device *coil)
 	 * will just use abstract codes here. In any case, smaller value = focus
 	 * position farther from camera. The default zero value means focus at
 	 * infinity, and also least current consumption.
+	 *
+	 * The two controls below control the current. The
+	 * FOCUS_ABSOLUTE is there for compatibility with old user
+	 * space whereas the VOICE_COIL_CURRENT should be used by both
+	 * new applications and drivers.
 	 */
-	v4l2_ctrl_new_std(&coil->ctrls, &ad5820_ctrl_ops,
-			  V4L2_CID_FOCUS_ABSOLUTE, 0, 1023, 1, 0);
+	coil->focus = v4l2_ctrl_new_std(&coil->ctrls, &ad5820_ctrl_ops,
+					V4L2_CID_FOCUS_ABSOLUTE, 0, 1023, 1, 0);
+	coil->curr = v4l2_ctrl_new_std(&coil->ctrls, &ad5820_ctrl_ops,
+					  V4L2_CID_VOICE_COIL_CURRENT,
+					  0, 1023, 1, 0);
 
 	if (coil->ctrls.error)
 		return coil->ctrls.error;
 
-	coil->focus_absolute = 0;
 	coil->focus_ramp_time = 0;
 	coil->focus_ramp_mode = 0;
 
-- 
2.7.4