[RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Hans Verkuil]

When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you know
when the new position has been reached? If this is controlled through a motor,
then it may take some time and ideally you would like to be able to get the
current absolute position (if the hardware knows) and whether the final position
has been reached or not.

In addition, it should be possible to raise fault conditions.

The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
anything about what it returns when you read it: is that the current absolute
position, or the last set absolute position? I suspect it is the second one.

If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
ditto for the other variants) that is a read-only control returning the current
position with the same range and unit as the ABSOLUTE control.

For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS comes
close, but it is too specific for auto focus. For manually positioning things
this might be more suitable:

V4L2_CID_IRIS_STATUS	bitmask

	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
	V4L2_IRIS_STATUS_FAILED

And ditto for the other variants.

Interaction between V4L2_CID_FOCUS_STATUS and AUTO_FOCUS_STATUS:

If auto focus is enabled, then FOCUS_STATUS is always 0, if auto focus is
disabled, then AUTO_FOCUS_STATUS is always IDLE.

Comments? Ideas?

Regards,

	Hans

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Hans Verkuil]

Ping!

I could really use some help for this. Any sensor API experts around who know
about how this is supposed to work?

Regards,

	Hans

On 07/02/15 14:00, Hans Verkuil wrote:
> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you know
> when the new position has been reached? If this is controlled through a motor,
> then it may take some time and ideally you would like to be able to get the
> current absolute position (if the hardware knows) and whether the final position
> has been reached or not.
> 
> In addition, it should be possible to raise fault conditions.
> 
> The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
> anything about what it returns when you read it: is that the current absolute
> position, or the last set absolute position? I suspect it is the second one.
> 
> If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
> ditto for the other variants) that is a read-only control returning the current
> position with the same range and unit as the ABSOLUTE control.
> 
> For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS comes
> close, but it is too specific for auto focus. For manually positioning things
> this might be more suitable:
> 
> V4L2_CID_IRIS_STATUS	bitmask
> 
> 	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
> 	V4L2_IRIS_STATUS_FAILED
> 
> And ditto for the other variants.
> 
> Interaction between V4L2_CID_FOCUS_STATUS and AUTO_FOCUS_STATUS:
> 
> If auto focus is enabled, then FOCUS_STATUS is always 0, if auto focus is
> disabled, then AUTO_FOCUS_STATUS is always IDLE.
> 
> Comments? Ideas?
> 
> Regards,
> 
> 	Hans
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Laurent Pinchart]

Hi Hans,

On Thursday 02 July 2015 14:00:23 Hans Verkuil wrote:
> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you
> know when the new position has been reached? If this is controlled through
> a motor, then it may take some time and ideally you would like to be able
> to get the current absolute position (if the hardware knows) and whether
> the final position has been reached or not.

There's only two drivers implementing pan and tilt (pwc and uvcvideo), one 
driver implementing focus and iris (uvcvideo) and three drivers implementing 
zoom (uvcvideo, m5mols and s5c73m3). Both m5mols and s5c73m3 seem to use the 
zoom control for digital zoom, so we can ignore them for this discussion. 
Where's thus left with pwc and uvcvideo. I'll comment mainly on the latter.

> In addition, it should be possible to raise fault conditions.

UVC specifies a way to implement asynchronous controls and report, through a 
USB interrupt endpoint, completion of the control set operation. This can be 
used by devices to report that physical motion has finished, or to report 
errors. Whether a control is implemented in a synchronous or asynchronous way 
is device dependent.

> The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
> anything about what it returns when you read it: is that the current
> absolute position, or the last set absolute position? I suspect it is the
> second one.
>
> If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
> ditto for the other variants) that is a read-only control returning the
> current position with the same range and unit as the ABSOLUTE control.

UVC doesn't explicitly define what a device should report. It hints in a 
couple of places that it should be the current position, but I believe it 
might be device-dependent in practice.

> For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS
> comes close, but it is too specific for auto focus. For manually
> positioning things this might be more suitable:
> 
> V4L2_CID_IRIS_STATUS	bitmask
> 
> 	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
> 	V4L2_IRIS_STATUS_FAILED
> 
> And ditto for the other variants.

Do we need to report that the control set operation is in progress, or could 
applications infer that information from the fact that they haven't received a 
control change event that notifies of end of motion ?

Failures need to be reported, but they're not limited to the controls you 
mention above, at least in theory. A UVC device is free to implement any 
control as an asynchronous control and report failures. Would it make sense to 
add a control change error event instead of creating status controls for lots 
of V4L2 controls ?

> Interaction between V4L2_CID_FOCUS_STATUS and AUTO_FOCUS_STATUS:
> 
> If auto focus is enabled, then FOCUS_STATUS is always 0, if auto focus is
> disabled, then AUTO_FOCUS_STATUS is always IDLE.

-- 
Regards,

Laurent Pinchart

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Hans Verkuil]

Hi Laurent,

On 07/08/15 13:25, Laurent Pinchart wrote:
> Hi Hans,
> 
> On Thursday 02 July 2015 14:00:23 Hans Verkuil wrote:
>> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you
>> know when the new position has been reached? If this is controlled through
>> a motor, then it may take some time and ideally you would like to be able
>> to get the current absolute position (if the hardware knows) and whether
>> the final position has been reached or not.
> 
> There's only two drivers implementing pan and tilt (pwc and uvcvideo), one 
> driver implementing focus and iris (uvcvideo) and three drivers implementing 
> zoom (uvcvideo, m5mols and s5c73m3). Both m5mols and s5c73m3 seem to use the 
> zoom control for digital zoom, so we can ignore them for this discussion. 
> Where's thus left with pwc and uvcvideo. I'll comment mainly on the latter.
> 
>> In addition, it should be possible to raise fault conditions.
> 
> UVC specifies a way to implement asynchronous controls and report, through a 
> USB interrupt endpoint, completion of the control set operation. This can be 
> used by devices to report that physical motion has finished, or to report 
> errors. Whether a control is implemented in a synchronous or asynchronous way 
> is device dependent.
> 
>> The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
>> anything about what it returns when you read it: is that the current
>> absolute position, or the last set absolute position? I suspect it is the
>> second one.
>>
>> If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
>> ditto for the other variants) that is a read-only control returning the
>> current position with the same range and unit as the ABSOLUTE control.
> 
> UVC doesn't explicitly define what a device should report. It hints in a 
> couple of places that it should be the current position, but I believe it 
> might be device-dependent in practice.

What does the uvc driver do today when an application asks for the IRIS_ABSOLUTE
value? Does it pass that request on to the webcam or does it return the last
set value?

We can choose to have the control type determine what the control does:

if it is just a normal integer control, then it returns the last set value. If
the hardware can return the current position, then it can set the new
V4L2_CTRL_FLAG_EXECUTE_ON_WRITE flag, which means that when you read it you get
the actual position, and when you write it you set the desired position.

But I don't know if the UVC driver can use this. I guess from your description
that it can determine this based on whether the control is synchronous or
asynchronous.

>> For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS
>> comes close, but it is too specific for auto focus. For manually
>> positioning things this might be more suitable:
>>
>> V4L2_CID_IRIS_STATUS	bitmask
>>
>> 	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
>> 	V4L2_IRIS_STATUS_FAILED
>>
>> And ditto for the other variants.
> 
> Do we need to report that the control set operation is in progress, or could 
> applications infer that information from the fact that they haven't received a 
> control change event that notifies of end of motion ?

You do need some sort of 'in progress' control or status. You don't want to
require applications to check control change events for the new value and compare
it to the end-value to determine if it is moving. And you can't use that anyway
if an error occurs that forces the motor to stop moving before the final position
has been reached.

> 
> Failures need to be reported, but they're not limited to the controls you 
> mention above, at least in theory. A UVC device is free to implement any 
> control as an asynchronous control and report failures. Would it make sense to 
> add a control change error event instead of creating status controls for lots 
> of V4L2 controls ?

Hmm, interesting, need to think about this.

Question: when you set an asynchronous uvc control, does it report back that
the new value was set OK, or does it only report back errors? And what sort
of errors are defined?

Regards,

	Hans

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Sakari Ailus]

Hi Hans,

On Thu, Jul 02, 2015 at 02:00:23PM +0200, Hans Verkuil wrote:
> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you know
> when the new position has been reached? If this is controlled through a motor,
> then it may take some time and ideally you would like to be able to get the
> current absolute position (if the hardware knows) and whether the final position
> has been reached or not.

On voice coil lenses it's also not possible to know when the position has
been reached, however you can estimate when it has happened based on the
intended movement and algorithm used to move the lens. This is far from
trivial though.

For low-level lends drivers knowing where the lens is is not feasible IMO at
the moment.

> 
> In addition, it should be possible to raise fault conditions.

Do you know of hardware that can do this? The only non-buffer related
devices that can do this I'm aware of are flash controllers.

I'd only define them if there's actual hardware that can actually raise
these conditions.

> 
> The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
> anything about what it returns when you read it: is that the current absolute
> position, or the last set absolute position? I suspect it is the second one.

I wonder if this has been really well thought out when the controls were
defined. I share your assumption.

> If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
> ditto for the other variants) that is a read-only control returning the current
> position with the same range and unit as the ABSOLUTE control.

This makes perfect sense if you have hardware that can tell the current
position.

Only one of the RELATIVE and ABSOLUTE controls should be implemented. I
guess no driver in mainline would implement both, I've seen such outside
mainline though. :-)

> 
> For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS comes
> close, but it is too specific for auto focus. For manually positioning things
> this might be more suitable:
> 
> V4L2_CID_IRIS_STATUS	bitmask
> 
> 	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
> 	V4L2_IRIS_STATUS_FAILED
> 
> And ditto for the other variants.

If you have hardware that can give you the information, sure.

> Interaction between V4L2_CID_FOCUS_STATUS and AUTO_FOCUS_STATUS:
> 
> If auto focus is enabled, then FOCUS_STATUS is always 0, if auto focus is
> disabled, then AUTO_FOCUS_STATUS is always IDLE.

Looks good to me.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Hans Verkuil]

On 07/22/15 10:21, Sakari Ailus wrote:
> Hi Hans,
> 
> On Thu, Jul 02, 2015 at 02:00:23PM +0200, Hans Verkuil wrote:
>> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you know
>> when the new position has been reached? If this is controlled through a motor,
>> then it may take some time and ideally you would like to be able to get the
>> current absolute position (if the hardware knows) and whether the final position
>> has been reached or not.
> 
> On voice coil lenses it's also not possible to know when the position has
> been reached, however you can estimate when it has happened based on the
> intended movement and algorithm used to move the lens. This is far from
> trivial though.
> 
> For low-level lends drivers knowing where the lens is is not feasible IMO at
> the moment.
> 
>>
>> In addition, it should be possible to raise fault conditions.
> 
> Do you know of hardware that can do this? The only non-buffer related
> devices that can do this I'm aware of are flash controllers.

If a motor is involved to move things around, then that motor can cause
failures that you want to signal. For example if something is blocking the
motor from moving any further, overheating, whatever. I hate moving parts :-)

> I'd only define them if there's actual hardware that can actually raise
> these conditions.
> 
>>
>> The way the ABSOLUTE controls are defined is ambiguous since it doesn't say
>> anything about what it returns when you read it: is that the current absolute
>> position, or the last set absolute position? I suspect it is the second one.
> 
> I wonder if this has been really well thought out when the controls were
> defined. I share your assumption.
> 
>> If it is the second one, then I propose a V4L2_CID_IRIS_CURRENT control (and
>> ditto for the other variants) that is a read-only control returning the current
>> position with the same range and unit as the ABSOLUTE control.
> 
> This makes perfect sense if you have hardware that can tell the current
> position.
> 
> Only one of the RELATIVE and ABSOLUTE controls should be implemented. I
> guess no driver in mainline would implement both,

That makes sense.

> I've seen such outside
> mainline though. :-)
> 
>>
>> For the status/fault information I think the V4L2_CID_AUTO_FOCUS_STATUS comes
>> close, but it is too specific for auto focus. For manually positioning things
>> this might be more suitable:
>>
>> V4L2_CID_IRIS_STATUS	bitmask
>>
>> 	V4L2_IRIS_STATUS_MOVING (or perhaps _BUSY?)
>> 	V4L2_IRIS_STATUS_FAILED
>>
>> And ditto for the other variants.
> 
> If you have hardware that can give you the information, sure.
> 
>> Interaction between V4L2_CID_FOCUS_STATUS and AUTO_FOCUS_STATUS:
>>
>> If auto focus is enabled, then FOCUS_STATUS is always 0, if auto focus is
>> disabled, then AUTO_FOCUS_STATUS is always IDLE.
> 
> Looks good to me.
> 

Thanks!

	Hans

Re: [RFC] How to get current position/status of iris/focus/pan/tilt/zoom?

[Sakari Ailus]

Hi Hans,

On Wed, Jul 22, 2015 at 10:51:18AM +0200, Hans Verkuil wrote:
> On 07/22/15 10:21, Sakari Ailus wrote:
> > Hi Hans,
> > 
> > On Thu, Jul 02, 2015 at 02:00:23PM +0200, Hans Verkuil wrote:
> >> When using V4L2_CID_IRIS/FOCUS/PAN/TILT/ZOOM_ABSOLUTE/RELATIVE, how do you know
> >> when the new position has been reached? If this is controlled through a motor,
> >> then it may take some time and ideally you would like to be able to get the
> >> current absolute position (if the hardware knows) and whether the final position
> >> has been reached or not.
> > 
> > On voice coil lenses it's also not possible to know when the position has
> > been reached, however you can estimate when it has happened based on the
> > intended movement and algorithm used to move the lens. This is far from
> > trivial though.
> > 
> > For low-level lends drivers knowing where the lens is is not feasible IMO at
> > the moment.
> > 
> >>
> >> In addition, it should be possible to raise fault conditions.
> > 
> > Do you know of hardware that can do this? The only non-buffer related
> > devices that can do this I'm aware of are flash controllers.
> 
> If a motor is involved to move things around, then that motor can cause
> failures that you want to signal. For example if something is blocking the
> motor from moving any further, overheating, whatever. I hate moving parts :-)

I don't argue about whether or not to tell it to the user, but can the
hardware tell that to the driver? If it can, naturally the user should be
told.

What I think would be nice to pay closer attention to at some point would be
voice coil lens controls. These are typically cheap and difficult to control
devices, as you apply a current that's going to move a lens rather than
telling it to be moved to a certain position. The higher level control is,
in my understanding, often more or less tightly coupled with the AF
algorithms. At the very least the existing drivers are quite low level
drivers. We should look what are the commonalities among those. But that'd
be a new topic likely requiring multiple RFCs...

This will quite probably look at least somewhat different in terms of
controls compared to current manual focus controls.

-- 
Kind regards,

Sakari Ailus
e-mail: sakari.ailus@iki.fi	XMPP: sailus@retiisi.org.uk